Google is picking up 9 types on 266 000+ pages. I expect the number to jump when the reindexing is complete. For example all our articles (300 000+) are marked as Item (Article), and is currently at 79 000. Since I took this print screen two days ago the amount indexed has increased by 2 000 items.
At the same time I looked at the crawling statistics, and noticed that the statistics have changed quite a lot. The amount of pages that Googlebot is crawling has gone up, and the response time has improved quite a bit. We also got 100 000+ more pages in the index now than we did before. The reason for the improved response time is due to the fact that we changed to a newer server.
The reason for the huge spike in pages Googlebot is crawling can have several explanations:
The 301 redirects from all our old articles (10 years) are being processed. We are like all sites seeing a Google referral traffic dip (total traffic is not down because it has been replaced by more direct requests) for the last three months due to all the URLs that where changed when we moved to the new platform.
Most likely Google is also indexing more of our older articles, that did not necessary make it into the Google index when served from the old platform.
Googlebot likes the new structure so much that it want’s to visit more often 😉
If you prefer, watch this recording of the presentation:
So why mobile first web app, and not an app or mobile specific site?
We knew we wanted to have a mobile first responsive design, so we talked to people about what their experiences where and became even more convinced that this is the way to go forward.
I recently read two good blogposts on the subject, one by Josh Clark
“First, a growing number of people are using mobile as the only way they access the Web. A pair of studies late last year from Pew and from On Device Research showed that over 25% of people in the US who browse the Web on smartphones almost never use any other platform.” … ”There’s a digital-divide issue here. People who can afford only one screen or internet connection are choosing the phone. If you want to reach them at all, you have to reach them on mobile. We can’t settle for serving such a huge audience a stripped-down experience or force them to swim through a desktop layout in a small screen.” – Josh Clark
It pretty much summarizes they way we at svenska.yle.fi look at it.
By designing the system (not site, I see it much more as an ecosystem when designing responsive sites) mobile first, and not hiding content from mobile users you are reaching a bigger audience. The same by not requiring users to download an app and keep updating an app. It is also less expensive to maintain, and one has to remember the cost of marketing the app so that users find it.
With an web app/mobile first site shared content is always displayed in the best way possible for the end user, depending on the device.
The perhaps most successful web app is Financial Times. It is much more device tailored than our site, but we embraces the same thinking.
When we look at our numbers, we have gone up from 8.5 % mobile usage (of total traffic) in January 2012 to 13 % in May. – svenska.yle.fi
Actually making it mobile first
Making the site mobile first sets some limitations, and also affects the design process. People are designing from mobile to desktop, or from desktop to mobile. In my experience it works best to work on all sizes at the same time, while keeping the technical limitations of mobile in mind (designing in the browser, read more in this slide by Edward O’Riordan@edwardoriordan (Front End United 2012 Amsterdam).
I am planning to try out if changing the weights of the CSS groups can join some of the compressed CSS groups we currently have.
We also ran into extra work with IE specific CSS files, as it is impossible to use weights to add them after the Omega CSS files. We ended up using more targeted CSS selectors for IE.
The amount of DIV’s is also something that is not optimal, as you can see in this image. More than half of the DIV’s could be removed.
Which theme, fluid or set of sizes?
The hottest theme in 2011 was Omega, and we decided to go with that one as we saw it as opportunity to get something off the shelf that was not perfect, but works. We wanted to focus our resources on making the solutions that were not available to us the best possible (and theme independent).
We decided to go with a set of sizes because it gives us better control over the layout.
I was asked a question during the presentation; ”What theme would you use if you would start from scratch with the knowledge you now have”. The answer is I don’t know, I would definitely look at what options are available – and most likely end up with a custom theme because of our specific needs.
We decided to go for a very sparsely decorated layout, to keep the UI clean and to make the basic responsive design as light as possible. We wanted to gain experience before we had built a swamp that it is difficult to dig yourself out of.
I also wanted to develop a new method, and embrace the possibilites – while leaving pixel perfect behind.
To do this I defined areas for graphics (at this stage the top of the page can be modified and display anything you like to make in Photoshop). This way I kept the layout completely CSS generated. An other defined area is for banners. They can be added anywhere, but we are still in the process of working out how they best are constructed.
The first thing we tackled were images. Fortunately we have our own homebrew Image Management System (IMS) that made it possible to get the images in many different sizes.
<img itemprop="image" alt="VPS, augusti 2011" title="VPS slog HJK." src="http://static.yle.fi/ims/imssv/2011/36/586284e6cf0a1e58bb/586284e6cf0a1e58bb_6.jpg" width="200" height="113" class="imsImg" data-ims-id="58628" />
<img itemprop="image" alt="VPS, augusti 2011" title="VPS slog HJK." src="http://static.yle.fi/ims/imssv/2011/36/586284e6cf0a1e58bb/586284e6cf0a1e58bb_5.jpg" width="200" height="113" class="imsImg processed" data-ims-id="58628" data-src-fluid="http://static.yle.fi/ims/imssv/2011/36/586284e6cf0a1e58bb/586284e6cf0a1e58bb_5.jpg" data-src-narrow="http://static.yle.fi/ims/imssv/2011/36/586284e6cf0a1e58bb/586284e6cf0a1e58bb_4.jpg" data-src-normal="http://static.yle.fi/ims/imssv/2011/36/586284e6cf0a1e58bb/586284e6cf0a1e58bb_6.jpg" data-src-wide="http://static.yle.fi/ims/imssv/2011/36/586284e6cf0a1e58bb/586284e6cf0a1e58bb_5.jpg">
To prevent the images from making the page flicker when the bigger image is loaded and inserted into the page we specified the image size in the CSS. This makes the image look blurry for a while when the page is loading, and then has a “sharpen” effect when the bigger image is displayed.
Video & audio
Most of our videos and audio are embedded from our media service Arena, and fortunately it was also being rebuilt at the same time. This meant that we got a Flash free first embed, that has a big effect on our mobile first approach and the rwd solution is out of the box. The solution could be improved by making the embed mobile first (the image is one size only). Flash is required for playback.
For Youtube and Vimeo we are using FitVids.js
KB size and http requests
The image and video solution means that the KB size was solved. One can argue that only one image file should be sent to the desktop users (instead of two), but I find the solution benefiting also them, because the image will display faster even though it is blurry.
To limit the amount of http requests, KB size and user experience (quick load for all sizes) we made the social media share buttons load only after the user on desktop hover on the text. On mobile they are loaded only after the user clicks on them.
So did it pay off? I think so 🙂
The data was collected with Yslow for Firefox.
The reason the subject pages are not lighter on mobile is that we currently do not have a breakpoint that would allow us to not load part of the page content on mobile (but offer a link like we do on articles).
In Views, always unclick to remove the extra markup – and give the block a unique style name so that your CSS is not dependent on block names.
We used a small image in our first iteration, and got feedback on that it was to small – so we changed that. Now we get some feedback that it is too big. I think think this is related to the fact that mobile is becoming the primary device, and you want to have the full version – you don’t want a preview that forces you to use a desktop to see the full image. So some more work needed to make all user types happy.
Iframes and auto height is causing grief. No perfect solution yet.
What to do next
Improve the theme / switch to a newer one
Improve on Twitter, Facebook integrations – they are not working correctly with the responsive design
As many of you already know we are using Drupal on the new svenska.yle.fi site. Some of you may also know that Drupal is an open source platform. Open source among other things means that users can use Drupal free of charge and more importantly be a part of the development process. Drupal is actually a very basic framework that does not have many out of the box functionalities, but instead offers a robust platform to work with to create solutions for your specific needs. Drupal’s core functionalities are meant to be extended with contribute (or contrib) modules in order to achieve the desired functionality for each site.
We here at svenska.yle.fi also want to be a part of the community and we are pround to announce our first public contrib module that is given back to the community. The module is called ONKI. ONKI links the Drupal taxonomies with the Finnish ontology service called Onki. Onki offers the needed ontologies, vocabularies and thesauri needed to publish content cost-efficiently on the Semantic Web.
The ONKI module uses mainly ontologies from Onki. Ontologies are conceptual models that identify concepts of a domain (like medicine, animals, plants) and they also contain the machine readable descriptions of the relations between the concepts (usually words). In Drupal the ONKI module is attached to the desired taxonomies and the module uses Onki to autocomplete results when the user enters a word they want to tag their article with.
Drupal has a procedure on how new modules are accepted into the community as contrib modules. More info can be found on Drupal.org (D.O.). In short the process an be divided into into three steps. First you have to create a sandbox project on D.O. where you develop your module. The sandbox project has most of the same functionalities (like issues) as a full project, but it is not found from the module directory nor will it have a clean URL. The development can be in any development environment but it is considered good practice to host your projects on D.O.
The third step is the one where your module is granted the privilege to become a full project. Full projects can be downloaded and used by any Drupalist, so full projects have to be maintained actively since for an example the Drupal core is constantly changing. Those changes can cause your module to malfunction. Also security fixes are important to be dealt with in a full project.
To summarize, Drupal is a self evolving and open community that offers an environment for developers to work on their projects. We here at svenska.yle.fi want to be a part of that community.
In an attempt to follow the ‘mobile first’ ethos, reducing the amount of loaded assets on smaller devices is high on the priority list.
Using media queries we can effectively control the loading of things like background images, but what about the CSS itself? After all, it seems wasteful to load CSS specifically for desktops when viewing a site on a mobile device.
A common misconception of media queries is that a device only downloads the CSS of a matched query (when using the query in the media attribute of a link element).
This is not the case.
All CSS files are downloaded regardless, though the CSS is not applied until you match the requirements of said query.
I feel like I’m being slightly cheated. In an ideal world I’d like the CSS to only be downloaded when they are needed, based upon matching the requirement of the media query. So how can I watch my http requests and keep my slimming mobile figure?
var importStatement = "";
case ($dc.screenWidth > 639):
importStatement = '@import url("http://some.tablet.css") screen and (min-width: 640px);' + importStatement;
importStatement = '@import url("http://some.compact.css") screen and (max-width: 639px);' + importStatement;
Dealing with Drupal
Our previous example was very basic, aren’t all examples?! When we consider taking this approach in Drupal we come across a few hurdles.
Drupal’s CSS optimisation aggregates and compress theme, core and contrib CSS together into a greatly reduced number of files. This is great, not least because there seems to be a huge amount of core and contrib CSS in Drupal.
But as soon as we do this we loose the ability to control our CSS for different devices with a little more finesse.
A module in the making?
Ideally there would be a module which would do the following:
Show a list of all CSS which is put out by the system, theme, core and contrib, preferably with a description as to its purpose.
Do you know where/what all of the CSS Drupal spits out is? Probably not. I’ve never sat down and gone through all of these files and figured out exactly what they do, and indeed if I really want them. On more than one occasion I’ve found myself writing CSS to override stuff from them. This shouldn’t be the case.
Allow selection/deselection of said CSS, removing what you don’t want.
An ability I’ve always longed for, yet it’s never been that important to warrant the time to actually figure out how to do properly (it’s generally been quicker to just override it in the theme). But, this is the ultimate, complete control on what styles are used.
Traditionally you’d simply override an unwanted style sheet in the theme info file by using an empty declaration with the same name. In some themes, most notably Omega, this feature is now offered as a theme setting. It allows you to switch off core and theme CSS.
Specify groups which are separately optimised.
Having Drupal optimise our CSS is a good thing, but as stated, it kind of walks all over our plans to deliver style sheets as needed rather than the whole lot together. Having the ability to set specific groups would be great. I could then lump all core/contrib stuff together and then have the needed separate—compressed—files for my ultimate goal!
I hereby name this module CSS Admin.
We’ve specified a useful CSS administration module there, and so in a modular mind set the features left to finish our solution should probably be rolled into another module which is just dependant on CSS Admin.
We need the ability to:
Perhaps we could call this CSS Admin – Media Query Loader.
That’s it. Simple!
Perhaps deferring the loading of CSS files is getting too picky, but I’d don’t think so. Granted, I’m probably missing a few angles of reasoning from this, but I think the core message is sound—we should be doing everything to honour the mobile first ethos by asking these kinds of questions and seeking out their solutions.
I’ve no doubt creating those modules is possible, I’ve worked with some excellent Drupal developers and I’m sure there are plenty more out there. With any luck we might get to look at this during this project and I’ll be able to follow up with a solution post, but for now I’ll make do with seeding the thought amongst you all.
Knowing my luck, in true Drupal fashion, a module already exists for that! I hope it does, but I’ve yet to come across it.
The areas that we have been working on are markup & semantics, http requests, amount of loaded KB and screen size adjustment.
On the markup and semantics front we have made additional switches from a traditional <div> structure to one of HTML5 tags (we where happy that the <time> tag was re-introduced). Our timestamps are printed in a humanized format, see timeago – a jQuery plugin.
With regards to keeping the amount of http requests down we use regular methods like sprites (depending on how big you have to make the sprite, it might actually also save a good deal on total KB size) and media querys combined with a script loader to only load relevant CSS styles and JS per device. @bengoodyear created a script for loading the share buttons only when the user indicates that he or she wants to use them. An additional nice feature is that on mobile you will be able to click the share buttons individually, thus removing the need to load them all.
Images has the biggest impact when considering amount of loaded KB. This is the reason why we load a small thumbnail (16:9) as the default image. If the screen size is big enough a bigger image will load automatically. To avoid a “jump” when the bigger image is loaded we added the size of the bigger image in the CSS, so what the end user might be able to see (depending on loading speeds) is that the image goes from being blurry to sharp. The right CSS for our set of four layout widths is applied with media query.
We built a custom solution as we have our home-brew Image Management System that we connected to Drupal. If you are using Drupals own image field, you might want to check out Responsive images module.
With regards to the amount of loaded CSS files I tried out Responder module – a good solution if you want to add a responsive design to a current theme. In our case since we are using Omega Responsive HTML5 theme and have added our own solutions it does not look like a module we will add.
The markup generated by Drupal and its modules is not the best, there is a lot that does not need to be there, thus adding both unnecessary KB weight, accessibility and styling issues. This is something we are still working on, and I assume we will have to make some kind of compromise.
Responsive videos is also a bit of a challenge. In our case we need to support Arenan, Youtube and Vimeo.
There is a new version of Arenan being launched next year, and we are working with the implementation of that one. A big improvement will be made on how it is embedded. It will now be added as a background in a div, that is replaced by a Flash player when the user clicks the play button. Sorry, no HTML5 video support yet due to rights management issues.
This means that we can also get a small size image for each video/audio clip, and re-size it the same way we re-size images.
Currently we are experiencing problems with the size of the video, when it actually starts playing. This is something we are working on together with the team building the new version of Arenan.
Right now we are looking at using Fitvids module on all iframes to add the scaling ability to Youtube, Vimeo and other embedded materials. Might be that iframe as a target tag is to broad 😉 We will have to test and see, would have been nice if Vimeo and Youtube added a class name to their embed code, so that you would be able to target them.
With regards to the set of four widths we are using, the styling has progressed nicely. It takes a while to start thinking mobile first, but when you get a hang on it it comes naturally. The interesting questions that arise then are what kind of differences do you want on the different devices? Do you want bigger images if you are using a tablet? These are the kind of questions we are thinking about, and will test on users to see what the final version will look like.
We have decided to work with responsive design and are in the process of setting the widths of the pages. I took a look at the screen resolutions our visitors are using at the moment.
The setup we are looking at right now would have four brackets:
Less than 740px (mobile or small tablet) – would see a fluid page
Less than 980px (tablets or notebooks) – would see a fixed-width page of 740px
Less than 1220px (smaller screens) – would see a fixed-width page of 980px
Wider than 1220px (big screens) – would see a fixed-width page of 1220px
When the current number of screen resolutions is applied to these brackets this is the outcome:
2,5 % = Less than 740px
2,2 % = Less than 980px
16 % = Less than 1220px
80 % = Wider than 1220px
These numbers represent 94 % of all visits to our site, the remaining 6 % are spread over 2,300 different screen resolutions.
If we would have a wider page for bigger screens, the result would be:
2,5 % = Less than740px
2,2 % = Less than 980px
49 % = Less than 1360px – would see a fixed-width page of 980px
45 % = Wider than 1360px – would see a fixed-width page of 1360px
It’s also possible to move the width bracket for smaller screen sizes, but that immediately creates a cascade effect. The tablet user is not happy when grouped with the mobile users, and someone using a screen resolution of 1192px is not happy being grouped with the users of a 768px screen.
Additionally, the numbers don’t tell the full story as they don’t take into account that a user might display the browser window in a smaller size than the screen resolution they have.
What brackets are you using, and any particular reasons why you chose them?
”You must be raving mad to even try to do such a thing”, she said. ”It is simply impossible to change so many things simultaneously.”
This was one reaction to what we are now attempting to do now. We are trying to rethink how we do web here at the Swedish-language division of the Finnish Public Broadcasting Company (Yle), or Svenska Yle, as we are generally known. And the list of things we want to reform isn’t short: the technical platform, the concept, the look, the organization of people who contribute, and the way we do our journalism. It is also true that we have small resources and huge ambitions, but hey, would it be worth it if it were easy?
So this is just an initial note of welcome to this blog. We will post here to try to open up and document these processes publicly, mostly because we are believers in not only the principles for open source software but also those in open design. This will be a record for us, but also for others who will attempt to do similar things in the future. One clear advantage of being open is that the quality of your documentation can increase dramatically.
SYND refers to one of the strands in this big re-think. It is the one where we are building a new content management system for Svenska Yle—one that will be based on the open source system Drupal. This blog will mostly follow the process of building that platform, although we will perhaps do some occasional detours into other larger questions relating to the reform, as well as other matters that keep us busy here at svenska.yle.fi.
The planning of this web reform started already last spring and we now have a team of six developers who have, since last Monday, officially gotten their hands dirty on the project. The first phase lasts five sprints (ten weeks) and we have scheduled our first release for March 2012.
This blog will be used to showcase the work that developers and designers are doing, as well as to discuss business-related questions that we have been working on. Personally, I can pledge to post entries that try to shed a light on our main business goals and how we are thinking about our “roof concept.” Furthermore, you should expect some posts on how we are organizing the process and thoughts about why we chose open source and Drupal from both technical and business perspectives. Expect also to see lots of posts about Drupal in general and some about meta data, linked data and such things.
It is true: this is a huge thing for us and I think I can guarantee the ride will not be smooth, at least all the time. Please do join us!
SYND stands for “Svenska Yles nya Drupal-plattform,” which translates to the new Drupal platform of Svenska Yle. The acronym “SYND” means “sin” in English.
We will of course also be reporting individually about the project. These are the people to follow on the dev team:
Mårten Seiplax, Project Owner & AD @mseiplax
Tomi Mikola [Mearra], Project Manager @tomi_mearra
Christian Wikström, Developer @cwiks
Alexander Granholm, Developer @granholm
Ben Goodyear [Exove], Developer (Themer) @bengoodyear @typekitlove
Lauri Kolehmainen [Exove], Developer @dotsi
Some other key people related to the project:
Mikael Hindsberg, Concept Designer @mickhinds
Kristoffer Söderlund, Technical Producer @soderkr
Jonas Jungar, Svenska Yle News organisation @jonasjungar
Sami Kallinen, Head of programmes, Svenska.yle.fi @sakalli