Should We Test It a Bit More?

[Editor’s note: Do read the comments of this posting about Jokakoti.fi.]

Always the big question; how much should we test it? When is it ready?

When should we launch the product? Should we launch alpha or beta (private or public)? Is it ok to launch so called perpetual beta (take Gmail for example)?

Pretty good questions.

Lately I’ve read that one should always launch early (see Startups in 13 Sentences for one example).

Well, today I received an e-mail from Mtv3 inviting me to beta-test their new real estate service (jokakoti.fi). I can’t actually remember if I signed for invite – might have done that.

Anyway, used the link provided in the e-mail. Whee, HTTP-basic authentication set to protect the pages! Professional? Not so, IMHO. Typed in the general user name and password combo (was there an ad agency who came up with these?) – what follows?

Screen capture below:

MTV3's new real estate service, Jokakoti.fi as seen with FF 3.0.6, newest Flash installed (and working), running on WinXP

MTV3's new real estate service, Jokakoti.fi as seen with FF 3.0.6, newest Flash installed (and working), running on WinXP

Pretty nice, isn’t it?

Plain blue, small tick mark from Adblock showing that there’s some Flash present.

A quick peek at code;  almost valid HTML, awful conditional comments used for IE and yeah, that’s about it. Flash and Ajax, which don’t seem to work with my sturdy old computer.

Nice.

Accessibility? Can’t even evaluate it as I don’t see the content. Oh wait, I can, can I?

Yes I can: accessibility is not that good.

Lessons learned so far:

  • In order to use the site I have to have Flash 10 (latest version) installed.
  • Beta testing is a fine way to catch some bugs – as we see here;)

Small test with IE7 – no surprise – it works. (Screen grab below)
Pretty nice, though the slides are sluggish and so on.

jokakoti viewed with IE7

jokakoti viewed with IE7

Lessons learned so far?

  • not all users have the latest plugins installed
  • not all users have up-to-date computers (I’d reckon this works like a charm with my laptop)

Last but not the least, a small test with my favourite mobile browser, Opera Mini, using their wonderful Simulator/Demo. (Screen grab below, screen states that one has to have Flash and JS support).

jokakoti viewed Operamini simulator

jokakoti viewed Operamini simulator

Conclusions?

  • Developing cross-browser, accessible websites is not easy
  • One can’t rely on users having newest versions nor fast computers
  • Beta testing is worth one’s while.

2 thoughts on “Should We Test It a Bit More?

  1. Hello Ilkka!

    I’m one of the developers for the new jokakoti.fi site. I happened to see your blogpost and was happy to see that you’ve really thought about these things. Good feedback is valuable.

    First of all, as you said, you were invited to the “beta test” of this new service. I’d like to empasize the word “test”. We’re in no way thinking that this test is a final product similar to GMail beta or such.

    The main objective is to receive feedback on problems and suggestions. We have had plenty of good responses from the beta testers. Both praising the user interface and also pointing out those problems that one can expect to find on a work on progress. Based on this communication and other known missing things we’re daily working on the next version.

    You mentioned the http auth as being not so professional. Well, there’s a good reason to use it for now. The service is not going to be password protected from the start page onwards like GMail, instead it asks for password once user tries to use the user specific features, such as adding an home to sell. But for the beta test we wanted to limit the audience temporarily. For this reason we did not want to spend valuable time on building “more professional login features” that would be scrapped once the service is really launched. So for now it does what it was planned to do, provide an easy, cost effective way to “hide” the test from the world.

    You showed a blue screenshot. There is one glitch that has been fixed in the latest development version (not in beta test currently) that may have caused this, but it should happen only with flash player version between 9.0.0 and 9.0.28. Another cause might be too strict adblock (like you mentioned) settings or other non-standard browser plugin misbehaving. I’d like to know more about this problem, so that we can solve it. You can contact me personally if you wish.

    You also mentioned that you were not surprised that firefox failed and ie did work. Well, I was a bit amazed and I’d imagine that adblock was thinking that the site content was an ad or something, because I personally develop on Firefox, and test on a multitude of browsers, (opera, ie6, ie7, firefox 2, firefox 3 to name a few). So as I’m running Windows XP, latest Firefox and latest flash, just like you.

    This blog post also talked about “almost valid html”, conditional comments for ie, flash and ajax. Yes, the html is almost valid, there’s one missing “/” mark on the tracking image. That’s fixed in the latest development versions (not in beta), thanks for that, it had slipped past me. Other than that minor bug the page validates nicely(which can not be said for most of the websites).

    The conditional comments are there mostly because internet explorer quite often misbehaves. I noticed just after we opened the beta that those are not needed for the site currently, so those are gone from the latest dev versions too. In bigger sense, I feel that serving conditional comment fixes for internet explorer is a minor beauty spot compared to the other solutions to overcome the ie bugs (like non-standard html or css code to filter out other browsers).

    As this site is based on Flash and Flex technologies to give it’s users rich interface and fast searches, obviously flash objects are used on the site. This site currently does not use Ajax for anything. All the javascript included is used to enhance flash and flex functionality. If matching features would have been built with ajax technologies, we’d have more crossbrowser issues than now and it would probably work slowly. Also, providing all current features without flash and javascript is impossible with basic html.

    You mentioned that in order to use the site you need flash 10 installed. Sure, having the latest flash 10 does not hurt. But the real requirement is 9.0.28 as that’s what flex requires to work. It was released in 2007 and it or later version was installed on 98% of browsers on December 2008. (Source: adobe.com)

    There was also talk about Opera Mini, which indeed is a wonderful browser on mobile phones. Sadly it does not support flash. Also my original mobile browser from Nokia does not support flash 9.0.28, but version 8 instead. This is something that’s harder to change. At least we try to explain that one needs these things for the site to work. I agree that it’s really annoying. Sadly we can not fix phone browsers flash support. One solution for this could be building an simplified mobile site for mobile users. Something that many big sites do.

    Flex is a relatively new technology and makes great new things possible. With it we can provide a better way to handle many aspects of an old concept. And like with all new things, it has it’s shortcomings. We’re doing our best to avoid them and provide a good experience for the user. These technological choises have been considered carefully and it’s a choise that has not been made on a whim.

    All in all, I’d like to sincerely thank you for using time to evaluate the site and write this blogpost about your findings. I hope that my long reply has answered to most of the questions you raised on your findings and that you find the site enjoyable in the future.

    Thanks!

    Mikko Tapionlinna
    Jokakoti.fi developer at Uoma Oy

  2. Hi Mikko!

    I have to admit I’m impressed with your answers, thank you! You made really good points and yes, I was perhaps a bit quick and harsh on some of my remarks.

    Adapting new technologies like flex is not easy as you mentioned. Jokakoti, IMHO (when I got to actual service;)), was a fine example of rich internet application. No problem luckily there!

    And yes, nagging about login, conditional comments and so on is perhaps what we Finns might call “hiusten halkominen“, have to give you that. Your answers made the decisions clear.

    HTML did actually look good, which is really nice. Points to you!

    Opera mini part is a shame, of course the reason being the support for Flash. I personally use Opera Mini for browsing the Web quite a lot – including real estates. I’d be really happy to use Jokakoti.fi with my monile browser, too;) Is there a mobile version coming some time? (House locations, driving directions or so?)

    Gmail’s perpetual beta is not what I thought jokakoti would be, and great that it won’t be so!

    All in all, thanks for the comment, I really appreciated it (and it shows that you really care about the services you develop.).

    Ilkka

Comments are closed.