Understanding how imgur overcome the SEO problem from SPA

5 min readApr 19, 2020

In SPA, SEO problem is the major problem hard to solve from developers

Brief description of SEO in create-react-app

create-react-app command is commonly used in many web projects. The technique called CSR (Client side rendering) provides much flexibility in both server side and client side resources management. However, handling SEO (Search engine optimization) for CSR is hard. It is because the search engine crawler may not execute your javascript completely. Some people may use SSR (Server side rendering) for solving this problem. It may require lots of time to rebuild your application.

In this article, we will share our experience in how we can solving the SEO problem in CSR. We will also build an SSR client targeted for the search engine crawler later.

Testing our CSR web application

Not like another web page, we cannot directly use Google Chrome for testing. When you open a web application which is using CSR. Google chrome will execute your javascript automatically building the HTML element from the react virtual dom. That’s the reason why the view is the same as the website using SSR. However, if you look deeply from your source code. You may notice the source is not the same as you view, at least most of the HTML dom will not display in the source. In most of the cases, you may not find the same text where the text is displayed in the view on the source code.

We take Imgur as an example. It is an image sharing service. They share a lot of memes.

The user view of imgur

The source code of imgur (most of the elements are javascript tag)

Reminder, the search engine crawler cannot fully execute javascript due to some technical issues (source: Google Webmaster). if they cannot fetch the content in the webpage. How to rank your website? Let's see how imgur solve the problem.

Using Postman

Postman is a simulation tool for simulating HTTP requests by different custom settings. Postman only return the content from the server and it won’t execute the javascript. If we input the url from imgur to postman, we got this result:

default postman setting of page: https://imgur.com/gallery/UlghMHB

<!doctype html>
<html lang=en>
<head>
    <meta charset=utf-8>
    <meta name=viewport content="width=device-width,initial-scale=1">
    <title>Imgur: The magic of the Internet</title>
    <meta name=keywords
        content="funny, image, gif, gifs, memes, jokes, image upload, upload image, lol, humor, vote, comment, share, imgur, imgur.com, wallpaper" />
    <meta name=description
        content="Discover the magic of the internet at Imgur, a community powered entertainment destination. Lift your spirits with funny jokes, trending memes, entertaining gifs, inspiring stories, viral videos, and so much more." />
    <meta name=copyright content="Copyright 2020 Imgur, Inc." />
    <link rel=icon type=image/png href=https://s.imgur.com/images/favicon-32x32.png sizes=32x32>
    <link rel=icon type=image/png href=https://s.imgur.com/images/favicon-96x96.png sizes=96x96>
    <link rel=icon type=image/png href=https://s.imgur.com/images/favicon-16x16.png sizes=16x16>
    <link rel=apple-touch-icon-precomposed href=https://s.imgur.com/images/favicon-152.png> <meta
        name=msapplication-TileColor content=#2cd63c>
    <meta name=msapplication-TileImage content=https://s.imgur.com/images/favicon-144.png> <meta property=og:url
        content=https://imgur.com/ data-react-helmet=true />
    <link rel=alternate media="only screen and (max-width: 640px)" href=https://m.imgur.com/>
    <meta name=p:domain_verify content=834554521765408b9effdc758b69c5ee />
    <meta property=og:site_name content=Imgur />
    <meta property=fb:admins content=12331492 />
    <meta property=fb:admins content=12301369 />
    <meta property=fb:app_id content=127621437303857 />
    <meta property=al:android:url content="imgur://imgur.com/?from=fbreferral" />
    <meta property=al:android:app_name content=Imgur />
    <meta property=al:android:package content=com.imgur.mobile />
    <meta property=al:ios:url content="imgur://imgur.com/?from=fbreferral" />
    <meta property=al:ios:app_store_id content=639881495 />
    <meta property=al:ios:app_name content=Imgur />
    <meta property=al:web:url content=https://imgur.com/ />
    <meta name=twitter:site content=@imgur />
    <meta name=twitter:domain content=imgur.com />
    <meta name=twitter:app:id:googleplay content=com.imgur.mobile />
    <meta name=twitter:card content=summary_large_image />
    <meta property=og:type content=website />
    <meta property=og:title content=Imgur />
    <meta name=twitter:title content=Imgur />
    <meta property=og:description content="Imgur: The magic of the Internet" />
    <meta name=twitter:description content="Imgur: The magic of the Internet" />
    <meta property=og:image content=https://s.imgur.com/images/logo-1200-630.jpg?2 data-react-helmet=true />
    <meta property=og:image:height content=630 data-react-helmet=true />
    <meta property=og:image:width content=1200 data-react-helmet=true />
    <meta name=twitter:image:src content=https://s.imgur.com/images/logo-1200-630.jpg?2 data-react-helmet=true />
    <meta name=twitter:image:height content=630 data-react-helmet=true />
    <meta name=twitter:image:width content=1200 data-react-helmet=true />
    <script>
        dataLayer=[];var pbjs=pbjs||{};pbjs.que=pbjs.que||[]
    </script>
    <script>
        !function(e,t,a,n,g){e[n]=e[n]||[],e[n].push({"gtm.start":(new Date).getTime(),event:"gtm.js"});var m=t.getElementsByTagName(a)[0],r=t.createElement(a);r.async=!0,r.src="//www.googletagmanager.com/gtm.js?id=GTM-M6N38SF",m.parentNode.insertBefore(r,m)}(window,document,"script","dataLayer")
    </script>
    <link href="https://s.imgur.com/desktop-assets/css/styles.0403d5d66015afdd7c5c.css" rel="stylesheet">
</head>
<body> <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-M6N38SF" height=0 width=0
            style=display:none;visibility:hidden></iframe></noscript> <noscript>If you're seeing this message, that
        means <strong>JavaScript has been disabled on your browser</strong>, please <strong>enable JS</strong> to make
        Imgur work. </noscript>
    <div id=root></div>
    <script async src="https://www.googletagmanager.com/gtag/js?id=UA-6671908-15"></script>
    <script>
        function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","UA-6671908-15",{send_page_view:!1})
    </script>
    <script type="text/javascript" src="https://s.imgur.com/desktop-assets/js/main.92c2ba25716d75a25b16.js"></script>
</body>
</html>

You may notices that only meta tag and script tag is returned. It is because we simulate a browse request a page, imgur will return javascript for client side rendering. Much more, the meta tag did not contain the full description of this website. They did not provide the title and description for this page. Just the default placeholder.

Using rich-results test from Google Search Console

To simulate Google bot, we can use the rich results test from Google. If we input the same URL which was tested in Postman, the magic will happen

if you look at the source from the test, you may see lots of <iframe> are in the source. While this is not the same as the HTML dom we known.

<iframe sandbox="allow-scripts allow-same-origin" id="8424772ec4" frameborder="0" allowtransparency="true" marginheight="0" marginwidth="0" width="0" hspace="0" vspace="0" height="0" style="height:0px;width:0px;display:none;" scrolling="no" src="https://eus.rubiconproject.com/usync.html"></iframe>
    ...
<iframe sandbox="allow-scripts allow-same-origin" id="989c1adf28" frameborder="0" allowtransparency="true" marginheight="0" marginwidth="0" width="0" hspace="0" vspace="0" height="0" style="height:0px;width:0px;display:none;" scrolling="no" src="https://u.openx.net/w/1.0/pd?gdpr=0&amp;gdpr_consent=">
    </iframe>

In our understanding, the webpage should like this.

So, what is happening? why there are three versions of source code for the one webpage?

They use the technique we called dynamic rendering. Which means we serve different contents for different HTTP agent.

image from pinterest

Why dynamic rendering

Dynamic rendering content both benefits between SSR and CSR. We can solve our SEO problem with SSR. Meta tag and description can be inserted in SSR.

Much more,we can also keep using our redex, javascript query select (i.e. document.getElementById) in the CSR. Using Progressive webapp in the client side, etc…

In the next article, we will implement dynamic rendering in Firebase hosting

If you love this article, please give us a clip👏🏽 and follow our pages