Publish a local markdown file to sites at once by Java, Problem-Solving & Enhancement

Explanation of the problems and solutions when uploading markdown files to DEV Community and Medium and the way to enhance the program

Β·

7 min read

1. Related Post

  1. Publish a local markdown file to sites at once by Java, Usage

  2. Publish a local markdown file to sites at once by Java, Configuration

2. Introduction

We will explain the problems of using other methods instead of API to import a local markdown file to DEV Community / Medium.

3. DEV Community

There are mainly 3 ways to import local markdown file to DEV Community.

  • Copy local markdown file content directly to DEV Community

  • Publishing to DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» from RSS (if you have published your post on other sites)

  • Using API

We will explain the problems of the above method and the advantage of using API.

3.1. Copy local markdown file content directly

In DEV Community, it does not recommend using level 1 heading in the article.

According to this link,

When you create a post, your post title automatically becomes a level one heading and serves a special role on the page, much like the headline of a newspaper article.

In your post content, start with level two headings for each section (e.g. '## Section heading text'), and increase the heading level by one when you'd like to add a sub-section

If you have a level 1 heading in the article, it will show the below message.

image.png

As a result, before you import the markdown content to DEV Community, you need to change all the level 1 headings to level 2, level 2 to level 3, and so on.

3.2. Publishing to DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» from RSS

If you have already published the article on another site, you can import it to DEV Community by this function, for detail please check this link.

Using this method may have the level 1 heading problem mentioned above, and also have no code highlight problem, which will be shown below.

3.2.1. Missing code highlight

image.png

As shown in the picture above, the code highlight is lost after importing to DEV Community from RSS.

No language specification is in the code block.

image.png

But in the original article, we can find the language specification.

image.png

3.3. API

DEV Community provides API to handle blogs programmatically. I suggested using version 0 at the moment since the post article API only exists in version 0.

Using API can resolve the code highlight problem, but for the level 1 heading problem, we need to handle it manually. As a result, I added one more # for all the headings in formatMarkdownText method in the DevToUploader.java.

DevToUploader.java java public static String formatMarkdownText( String markdown ) { return markdown.replace( "# ", "## " ); }

Hence, whenever we use the program to upload the local markdown file to DEV Community, it will fix the formatting automatically.

3.4. Remark

3.4.1. 403 error when using API

Sometimes, 403 error may occur if you use DEV Community API in your program.

It may be due to the user agent, which is blocked by DEV Community, since it regards you as a web scrapper program.

To solve this issue, changed the user agent to Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0 or others. (As shown in the example below.)

HttpRequest request = HttpRequest.newBuilder( URI.create( "https://dev.to/api/articles" ) )
            .header( "Content-Type", "application/json" )
            .header( "api-key", Token.getDevToApiKey() )
            .header( "User-Agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0" )
            .POST( HttpRequest.BodyPublishers.ofString( jsonStr ) )
            .build();

This link explains the causes of 403 error very well.

4. Medium

Medium does NOT support copy and paste markdown file content directly.

2 ways are provided to import article to Medium.

  • Import a story

  • API

4.1. Import a Story

Import a story means importing an article that you already published on another site. However, this method often leads to a disaster in formatting😒.

For example,

  • Lost of indentation

  • Absent of code block

e.t.c.

4.2. Using API

Using API can solve most of the formatting problems, except the no code highlight problem.

e.g. I published temp/sample.md to Medium by API, and the code highlight was lost.

image.png

If you copy the code to the code block, the indentation will be lost and you also need to select the programming language manually.

image.png

To resolve this issue, I have done the following in the program:

  1. Upload all the code blocks to Gist. (We need to convert the code block language to file extension when uploading to gist. That's why we need language2Ext.json)

  2. Get the gist link of the code and replace the code block with the gist link

  3. Medium automatically embeds the gist links in the article and the code highlight is preserved

Here is the result:

image.png

4.3. More formatting automation

Medium has the following behaviors when we upload the article by API.

  • Medium will treat the first image as the cover image of the article

  • Medium will treat the first line as the title

  • Medium will treat the second line as the subtitle

  • To reduce the formatting problem of embedded gist link, I suggest adding an empty line before and after the gist link

To achieve a better format by using the behaviors above, we format the markdown manually before upload.

public static String formatMarkdownText( String markdownText, Map<String, String> idToGistLinkMap, BlogInfo blogInfo )
{
    String formattedMarkdownText = markdownText;
    for ( Map.Entry<String,String> entry : idToGistLinkMap.entrySet() )
    {
        String id = entry.getKey();
        String gistLink = entry.getValue();
        formattedMarkdownText = formattedMarkdownText.replace( id, "\n" + gistLink + "\n" );
    };

    StringBuilder sb = new StringBuilder();
    sb.append( "\n![image.png](" + blogInfo.getImageUrl() + ")\n"  );
    sb.append( "\n# " + blogInfo.getTitle() + "\n"  );
    if ( ! StringUtils.isBlank( blogInfo.getSubtitle() ) )
    {
        sb.append( "\n# " + blogInfo.getSubtitle() + "\n"  );
    }
    sb.append( formattedMarkdownText  );
    return sb.toString();
}

image.png

5. Enhance functionality

5.1. Support more code highlight

If your code block is not highlighted in gist code and there is no file extension in the gist code file, please add the relevant language to file extension mapping in language2Ext.json.

You can add more mapping in the file if you want to enhance the program.

5.2. Upload to another blog site

If you would like to upload the markdown file to another blog site, you may follow the steps below.

  1. Add the site name in the enum SITE in BlogInfo.java

     public enum SITE {
         DEVTO, MEDIUM, NEW_BLOG_SITE
     };
    
  2. Add the new newBlogStatus in PUBLISH_STATUS for the new site

     public enum PUBLISH_STATUS {
         DRAFT("draft", "false", "draft"), UNLISTED("unlisted", null, null), PUBLIC("public", "true", "public");
    
         public final String mediumStatus;
         public final String devToStatus;
         public final String newBlogStatus;
    
         private PUBLISH_STATUS(String mediumStatus, String devToStatus, String newBlogStatus) {
             this.mediumStatus = mediumStatus;
             this.devToStatus = devToStatus;
             this.newBlogStatus = newBlogStatus;
         }
     };
    
  3. Add the uploader class and the test method to com.blog.publish.publisher.utils in src/main/java and com.blog.publish.publisher in src/test/java respectively

  4. Add a new implementation of the interface UploadArticle for the new blog site in App.java

     private static interface UploadArticle
     {
         public void exec( BlogInfo blogInfo, String markdown ) throws IOException, InterruptedException;
     }
    
  5. Add a new entry in siteToUploadArticleMap with the site name defined in step 1 and the interface implementation defined in step 4.

     private static Map< SITE, UploadArticle > siteToUploadArticleMap = Map.ofEntries(
         Map.entry( SITE.DEVTO, devToUploadArticle ),
         Map.entry( SITE.MEDIUM, mediumUploadArticle)
     );
    

5.3. Add command line option

I used commons-cli to handle command line option

<dependency>
    <groupId>commons-cli</groupId>
    <artifactId>commons-cli</artifactId>
    <version>1.5.0</version>
</dependency>

Existing command line options are defined in getCommandLineBy method in App.java. You may add/remove the options according to your need

6. Reason I wrote this blog

As my blog has more and more articles, I was wondering how I could promote my blog so more people can see it. The part "Key Mistakes a New Bloggers Make" in the article How I Made $30,000 From Writing in 1 Year has provided me a good insight. The author mentioned

Not writing everywhere at first (Hashnode, Dev. To, Medium, LinkedIn, Twitter)

I should post the articles to more sites so more people can access them. My strategy is to choose a base blog site (in my case, Hashnode), then published those articles again in Medium and DEV community, and finally created posts on LinkedIn and Facebook to advertise my blog regularly.

When I first tried to copy my original blog in Hashnode and pasted it in Medium, but the pasted blog has been badly formatted and it took more than a few minutes to fix it. So I wonder, is there an efficient way to publish my blog in several sites without breaking its formatting? πŸ€”

The article Programmatically Publish a Markdown File as a Medium Story With Python offers a solution that we can publish the blog to Medium through its API, which can fix most of the format problems. I checked Hashnode and DEV Community as well and API can be used in them also.

I optimized the code provided in the above link and wrote my first version in python. After that, due to some personal reasons, I wrote the command-line application again in Java since I would like to deepen my understanding of it.

Did you find this article valuable?

Support Ivan Yu by becoming a sponsor. Any amount is appreciated!

Β