Ip's. » survey

Nokia (Yet Again) Chosen as “Best Web Page in Finland”

Ilkka — Tue, 25 Aug 2009 12:25:03 +0000

Last February I wrote about Nokia’s site chosen as “Best Web Page in Finland“.

I was skeptical then and I’m skeptical now as the same media, Digitoday, publishes almost the same piece of news. Except that the company who did the research is different.
(Article from other source in English)

At march it was Webmark who did the research, now it’s Blue White Partners.

Anyhoo.

Article (in Finnish) claims that “Blue White Partners went through over thousand Finnish corporate web sites evaluating their marketing efficiency“. Jolly good.

Marketing efficiency?
Over thousand corporate web sites?

Hmm.

Well, as I last time learned, our journalists can be quite busy sometimes. So off to BW Partners we go to find the press release!
And yes, there it is ( press release as .pdf)

Whee, this is nice.
According to press release, methodology of the said research is based on “an average efficiency to attract visitors and turn them into customers, based on hundreds of thousands web sites evaluated“.

Errm, what?

And how the hell does one measure that?
And hundreds of thousands web sites evaluated?

Hmm.

Hey, there’s even more!
Again the Press Release states that “the average Blue White Score for Finnish Companies” (sic!) ” is 42, still below the international average of 50″. Now, Nokia, the front-seat student of all Finnish web sites, scores 99,8 and release states “which means that only 0,2 % of companies worldwide have as good or better web site as Nokia“.

Phee.

It seems that the rating goes from 0 to 100, 0 being the lowest score and 100 the highest. 100 obviously means “The Best”, there is no better and 0 that there is no worse site. (Please do correct me about the rating if I’m wrong)

The top-100 list for Finnish companies can be viewed at BW Partners Web site. All sites, listed in order of ranking, followed by score and change since the last evaluation.
(Small side note: depending on source about 8 % of (Finnish) men are red-green colour blind. If someone wants to indicate something, let’s take change, with colours, do use some other colours than red and green. Or better, don’t use colours alone, use something else. Like a minus-sign in front of negative changes?)

Anyhow, forward.

What’s nice is that I can purchase the evaluation report for only 295 eur!

Pretty cheap, isn’t it?

Conclusion and remarks

As such the evaluation portrayed here is worthless (at least to me). I don’t know the methodology, I don’t know the sample and so on.

Perhaps they’ll tell in the 295eur-report the following facts left missing in the press release and articles:

Exact size of the sample (“over thousand” is not that exact, you know. 1001 is over thousand, isn’t it?)
Methods of evaluation (“average efficiency to attract visitors ” and so forth is not that convincing)
Sites Sampled (“over thousand Finnish corporate web sites” doesn’t tell that much.)
Pages Sampled from every site or the algorithm for selecting the sample (did they evaluate only the front page or what? Did they start from the root and continue to leafs?)

Most likely the evaluation will be covered heavily by Finnish media, which is somehow sad.

Still waiting for the first proper “Best Web Site in Finland” evaluation.
I’m not holding my breath, though.

(Usual disclaimers apply; I do realize that evaluation methodology can be a trade secret, as well as other data. I also realize that business is business and everyone must make a living. )

[Edit 15.45, Arctic Startup has written about the first evaluation done by BW Partners. According to linked article the evaluation score is "is made by investigating content semantics, language, search engine rank and index, and presence in social media."]

Nokia’s Site Chosen as “Best In Finland”

Ilkka — Thu, 19 Mar 2009 11:52:32 +0000

[Update: I was contacted by Webmark about the survey. Read Updates at end of post.]

According to various (finnish) sources (Tietoviikko, Digitoday, Tietokone etc), Nokia.fi was chosen as “best web site in Finland, 2008“.

I don’t even want to begin to conversate about Nokia.fi’s pros or cons – I’ll just focus on how this achievement was published in the news.

Well yeah.

I spotted the news at Digitoday. It states that Webmark, “a company that measures quality of web sites”, found Nokia’s website the best in Finland (according to their evaluation/survey).
Then Digitoday, as well as all other publishers, continues jabbering about survey/evaluation done, which sites made it to Top-10 and so on.

Nice.

Except that:

I didn’t find a single link to survey/evaluation made
I didn’t find a single line mentioning tools used in the evaluation
I didn’t find a single line mentioning source for this bit of news
(Where did it come from (Heaven?)? Was there a press conference (where? when? who was invited?)? Did the company post a press release?)

Jolly good.

Now, as I’m by nature mean, nasty and curious, I googled Webmark first. Found eventually their finnish site. And yes! There it was, the press release about “Best Site in Finland, 2007″.

Wait, what?

2007? It should be 2008 now, shouldn’t it?

No press release about year 2008, just 2007.

Nice.

Well, back to Google.
Tried “webmark suomen paras verkkosivu“. No source, no press releases from original source, just different posts with (virtually) same content. (Other queries returned same results, too.)

Back to original post at Digitoday. CEO’s of Webmark and Accenture are mentioned, as well as that “at the same time published research done by Accenture” – hey, I’ll just see Accenture.fi for origins!

Nothing at all about these things at “press” -part of their pages.

Makes one wonder.

Hey guys at Webmark, Accenture, Digitoday and so on, if you read this, please post (all or at least one of the following):

original press release
origin of this news (if it was published at conference or something, where and when?)
actual evaluation data
methods used at evaluation

.. and then I’m willing to believe Nokia.fi is “the best web site in Finland, 2008″.

Update 25.3.2009

I was contacted by Webmark about this post and spent interesting times discussing the survey and methods. I also received some additional material, which made some points mentioned above quite clear.

Original press release was published at the given date at Webmark’s site, only in the afternoon (read it here, pdf)
Announcement was made at “Vuoden verkkosivut 2008″ -conference, 19.3.2009. (See the program here.)
Why didn’t I find the link nor the page? Only mention at Webmark’s front page about the announcement and conference (symposium?) was a flash -based banner, which got caught in my Adblock (it’s not a good idea to name banners “banner”, you know.). And when it comes to no origin at news about survey, thank our beloved and overworked reporters!
Evaluation data is not available, but is based on methodology by Mr. Harri Oinas-Kukkonen from University of Oulu.
See point 3 and press release.

(Bit more later…)

So I Took the Survey.

Ilkka — Tue, 29 Jul 2008 12:00:43 +0000

So I took the survey (A List Apart survey for people who make websites). And although I don’t currently do web sites for living, I do consider myself a web professional, too.

I Took the survey (A list apart survey for people who make websites)

Do take the survey!

(Don’t seem to find option for uploading pictures to WordPress if they’re not in a posting/page. Does it exist?)

Helsinki, UN and Certain Surveys.

Ilkka — Tue, 08 Jul 2008 12:44:05 +0000

United Nations (UN) just released their survey about municipal websites, world wide. Actual title was “Digital Governance in Municipalities Worldwide (2007) ~ A Longitudinal Assessment of Municipal Websites Throughout the World” (PDF-link!).

What was the survey about? (Bolding by yours truly, quotes from the summary part of the survey) In the survey they “evaluated the websites of municipalities in terms of digital governance and ranked them on a global scale”. Cities and nations were selected as follows: “The top 100 most wired nations were identified using data from the International Telecommunication Union (ITU)…”. and “The largest city, by population in each of these 100 countries was then selected for the study and used as a surrogate for all cities in the respective country…”

To summarize: top 100 most wired nations were under inspection, of which biggest cities’ websites were evaluated. Helsinki is the most crowded city in Finland, so Helsinki’s website (www.hel.fi) was evaluated.

Five components were evaluated:

Privacy/Security;
Usability;
Content;
Services; and
Citizen Participation

I’ll leave other aspects alone and concentrate on the second component, usability. On the survey they examined “three types of websites: traditional web pages, forms, and search tools”. Sounds like a good selection to me, although in my opinion those are just parts of one website. What was then, in fact, evaluated? Branding, structure, consistency (colours, navigational elements etc.), requirements stated (clearly), availability of sitemaps and alternative versions for documents, basic forms’ usability, search tools and so on. Quite good a selection (imho), again.

In the content-component they also evaluated the “Bobby compliance” (sic!) or access to web site via a TDD Phone Service. (Side notes: Bobby was originally “a free online tool provided by the Centre for Applied Special Technology (CAST) used to validate websites for WAI and Section 508 compliance” (Wikipedia). At 2005 it became “Watchfire WebXACT” -tool (still free to use online) and was finally integrated by IBM at 2008 and is nowadays part of IBM’s Rational Policy Tester Accessibility Edition and therefore not available for free use.)

In the usability section Helsinki was ranked fourth overall and in the content section sixth overall. Not too bad, eh? About Helsinki was said: “Helsinki increased in its overall score and its ranking
significantly from those in 2005 and 2003. Helsinki was ranked 35th with a score of 34.62 in 2005 and was not ranked in the top five cities in any of the categories.” Now Helsinki is among top ten in four out of five categories (privacy, citizen participation, usability and content – if I’m not wrong).

Hooray for Helsinki and hooray for Finland, as the survey quite clearly states that “(The largest city, by population … was … ) used as a surrogate for all cities in the respective country“. Points for Finland, not for Helsinki, I’d say, although finnish media wanted to state so (for example Helsingin Sanomat says that “Helsinki’s Website is the third best in the world” / (HS: Helsingin verkkosivut ovat kolmanneksi parhaat maailmassa).To claim that Helsinki’s site is the 3rd best in the world is at best misleading and at worst just plain wrong.

Anyway, pretty nice results.

Why am I not pleased?

Helsinki’s site is in my opinion not as good as all this hype could lead us to believe. Good, yes, they’ve done a lot of work, but still! The URI structure is awful (take http://www.hel.fi/wps/portal/Helsinki?WCM_GLOBAL_CONTEXT=/Helsinki/fi/Etusivu as an example), layout is table-based, front page has a lot of HTML errors and so on. Quite minor things, have to give you that.

The thing I’m worried about at the moment is that the given UN Survey will now be used as a way to measure basic quality of web sites in Finland (“Survey says that www.hel.fi is the third best in the world!”) and the not-so-nicely-implemented features will be forgotten (at hel.fi and other sites, too). Quality of finnish sites has improved but is not that good when accessibility and usability are taken into account. Even technical quality can be questioned in many cases.

To sum things up: Helsinki (or actually Finland) scored pretty well on municipal websites survey. Overall quality is ok, but in my opinion not as superior as the media tends to say. we still have a lot to do!