What’s not to like?

About 8 months ago, we added a pair of radio buttons to all the Flex 4.5 doc and ActionScript language reference pages that asks you is the content was helpful:

What we were hoping for was a large amount of data, good or bad, that would tell us where weaknesses and strengths exist in the documentation. This would give us much more insight into the efficacy of individual pages, or into groups of pages. It would also help us decide where to put our resources when updating the documentation. Unfortunately, the number of ratings has been extremely low. Like lower than many banner ad click through rates. Like lower than my Angry Birds score kind-of-low.

So, why aren’t you rating the pages?

Are you too busy as a developer to rate the page?
Maybe you’ve got a lot on your mind when you end up at a Flex doc page, and don’t want to distract from your thought process, especially if you’re copying/pasting a code example or trying to grok something.

Are you worried about privacy?
There is absolutely no way for us to track your rating or any comments you add with that rating. It is completely anonymous.

Is the location of the rating “widget” inconvenient?
The widget currently appears at the top of the page. But Flex doc pages tend to be long and dense with material, so maybe you see it and then just forget about it. We’ve thought about moving the widget to the bottom of the page, or adding it multiple times to the pages, but we don’t want to get in your way, either.

Do you think we ignore this data?
We don’t. We look at it very closely, and even read the comments.

We’d love to get your feedback (here, and, of course, on the help pages)!

-Matthew J. Horn
Flex doc team

17 Responses to What’s not to like?

  1. I use it, when I really feel was helpful or when I found the complete solution solution.

    But when I didn’t find what a looking, I didn’t rate because sometimes I understand I’m making wrong my search about the topic. My keywords aren’t correct.

    Be or no be helpful a topic it’s relative, I mean depends about my needs!

  2. Jon says:

    It’s a mindset thing. When I’m looking something up, I’m not interested in all the chatter. Don’t even see it. It’s like when you go to a website and have to click away the “research” popups.

    Besides, on any one day I’ll see dozens of AS3 pages, if not more. If I had to rate them all, I’d never get anything done.

    In the past I’ve added comments to help pages now and then, if I know they are in error. But having seen that nobody else ever uses that feature (and probably doesn’t read those comments either) I stopped doing it.

    I wouldn’t mind if Adobe collected some stats on my in-help searches and asked me specific, targetted questions about usage.

  3. Nikita says:

    For me it’s the problem of a decision between information and experience. Usually, the information is quite helpful, but I don’t like this new Adobe help system, it seems not productive, intuitive and stands on my way. I liked older, Macromedia ones. I liked built-in Eclipse help system more.

    So, though the question is precisely about the information I get, I perceive it as a question about overall experience and every time I find the information I feel frustrated a bit, so that I don’t feel like pressing “Yes” because I didn’t like the way I reached the info, and pressing “No” is also not an option, because info was indeed helpful.

  4. Hi,

    I believe the position is not ideal. It’s right before the first very important information : the class description. At this point, you only scroll to get to what you need. But even if you put it at the bottom of the page, it won’t get noticed.
    I don’t believe I have ever voted, even if I saw these pages like a few hundred times.

    And the question is a bit awkward. When you look at the docs, you are not looking for help, you are looking for plain information. If you need help, you go ask a question in the forum. Forums have (inc. Adobe ones) have this kind up “thumb up” or “star” feature that make sense in that context and target you to the helpful answer.

    If I was you, I would remove this widget and keep the comments, but I would maybe add a link to the Adobe forums so that user can start a thread.


  5. Trevor says:

    I think your biggest problem is by far discoverability.

    For one, its at the top of the page. Nothing is helpful yet because you haven’t even gotten to any content. Additionally, much of the content in that area is low-priority content, easily ignored by the viewer of the page. For example, in the AS LR, that area contains class name, the language version and runtime version(s) – most of which most viewers would already know.

    Then you have the fact that this control isn’t even displayed until all other content within the page (and subsequent frames) is loaded. I imagine most people, as is the case with myself, have already scrolled past its location on the page before it even shows up. In fact I had no idea it existed until this post. Even then it took me a while to find it due to this very reason.

  6. Because they are radio buttons. Do something better and I’ll rate more.

  7. DaveW says:

    Like Jon, I just don’t really notice the radio buttons, and even when I do I’m usually too busy trying to get some information and get back to work. Also, I just checked out some pages now to see where the buttons appear, and noticed they don’t show up right away. Perhaps people scroll down to the properties/methods before the radio buttons even show up.

    I do generally find the docs to be helpful though, and I like the online docs vs. local docs since they’re more up to date. The recently added quick search feature is great too.

  8. Victor Mendiluce says:

    I havent use it because the overall quality of the help system is so low that I don’t see the point in rating if the info in itself was useful or not.

    The help system is flawed, it is frustrating, and the user doesn’t have the feeling of any chance of improving it because it is sooo far of being a good system.

  9. Shirish says:

    IMO, why don’t you add a Oracle website’s feedback like widget. That widget follows user while page scrolling. Its not annoying but it is also hard to ignore.

  10. Rick says:

    I think you should take it as a you are doing an good job with the docs. Remember a non-answer can still be an answer.

    I think only the extreme ends want to be vocal. Holy crap this was a good doc I’m going to rate it! vs Holy crap this doc was terrible, who do I flame for this pile of dung?!

    While everyone else gets what they need and goes on their way.

  11. Terry says:

    But you *did* learn something: those radio buttons don’t work as a way to measure the effectiveness of your documentation.

    People tend to act out of self interest. Helping you — Matthew J. Horn — do your job doesn’t necessarily help your customers in any quantifiable way.

    So you need to either invent a pill which changes human nature, or you can provide your users with an incentive which is mutually beneficial.

    A suggestion:

    For as many pages as possible, add sample code which demonstrates how to perform an operation — for example, code which shows how to load an external swf (and how to unload it without causing a memory leak), or code which shows how to display a local external HTML page in an Android or iOS app (the code for this is not as simple as you might think — try it). Then add a button which, when clicked, will copy the formatted code to the user’s clipboard.

    Track how often those Copy buttons are clicked. This will help tell you if the information was useful.

  12. Mark Lapasa says:

    I would not be where I am today as a long time Flash Platform Developer without the excellent Macromedia/Adobe documentation. That said, I find rating anything is just an obstacle to what I really want, which is API information. Also, the nature of the ratings widget is useless in that if you find what you wanted, that’s great. As a dev, your off to the next problem you need to solve. There is no debriefing “How as your Adobe Docs experience today”. If you don’t find what you want, as a dev you don’t waste any time in trying to find what you need to find. The rating widget only slows the achievement of this objective. I find having “qualitative” comments at the footer of the pages is a good way to attract your team’s attention in terms of spelling errors, typos, or section of paragraph that don’t make sense…. oh yah… keep up the good work guys!

  13. Delfeld says:

    There are two points I would make about the ratings.

    I don’t like having to log in to do any research. I actually do not know if the rating is anonymous or not, since every other aspect where I could voice my opinion, I have to go to a separate page to log in, and it may or may not get me back to the original page! So I am certainly not going to go through the bother of *any* type of voting unless it is clearly no impact to my time. However, see the prior comments by others explaining why the yes/no vote is useless in the docs.

    The rating should be integral to the comments I would make. For example, StackOverflow.com encourages me to answer questions posed instead of simply rating. If I simply agree with an answer, since I am already ready to respond to the question, I actually see it as a great boon to be able to just add my vote, rather than explaining a problem. If I see an answer that needs improvement, I can add a sub-comment, and do not really need to vote. In this way, voting is integral to my response, and not a separate activity.

    I prefer to comment. But, as mentioned previously, the comments on docs seem to be incorporated very slowly, if at all, to the docs. I think that if you look around at all the blogs and boards talking about Adobe products, you will find a dynamic discussion occurring, usually with a succinct resolution. But there is no discussion at all on Adobe’s sites. It’s Adobe saying what is, and it makes Adobe seem archaic. If Adobe is going to print it as the Final Word, then a live chat (with real-time corrections being made to the docs by Adobe techs) would be the ideal scenario.

    I have written that idea to Adobe before. So why bother telling me that you “even read the comments”?

    All that said, I concur that I couldn’t have done it without Adobe docs. But I Google first, and then refer to the docs only when I have no other recourse.

  14. Delfeld says:

    And maybe allow me to edit my own comments. . . .

  15. ktu says:

    The problem is that information in the docs is helpful, and if you don’t think it is, you are ignorant and lazy. The documentation for flash has always exceeded my expectations and has always been better than any other language documentation.

    I believe the biggest issue is that all documentations are fused into one website that uses heavy js (that sometime crashes my browser) and makes it frustrating to get started using. (once the filters are in place its back to how it was a few years ago thank god).

    IF you want people to be more likely to give input:
    – separate the products. Stop lumping them all together. Make a menu that lets you switch products. Seriosuly, I want people to tell me how often they need to be able to browse more than one API at a time. A simple menu to switch is all that’s needed to make the experience better.
    – get the documentation back into the IDEs. I hate leaving my IDE to get the docs. Here’s how feedback occurs: in my IDE, I open the help, find waht I’m looking for, don’t need to switch software, try the example or implement a solution. If it works, great, if not, I can more easily switch back (maybe just shift focus to a different panel) and press a button saying that it wasn’t good.
    – as I read in the comments above, most people (that I can tell) avoid Adobes forums. Why? It doesn’t have to be that way. So the help panel also becomes a viewer for the forums and I can directly invoke a new thread page whenever something goes wrong.
    – lastly, the help panel should only comprise of html with minimal requirements on js & css. I want to be able to integrate the docs into any ide.

    Or, you could just:
    – stop coddling us. You’ve done great work on the documentation, treat people like idiots and they’ll stay idiots. Sit back, relax.

    To anyone else reading this. I solve >90% of my AS3 problems by using the docs alone. They are great.

  16. Dan says:

    When I am referring to the docs it is because I need more information about a problem I am trying to solve, that is not immediately solvable from my memory/knowledge, or the code complete in FB. My mindset at this point is concern to solve the problem, as I am now in a position where I have to invest more time than I originally hoped. When I solve the problem, I am relieved and just want to get back to work. I feel rating an invidividual element of ASDocs is like your phone asking you how your experience was every time you use the address book, or Google asking you how your experience was when you carry out a search.

    On the whole I think the ASDocs are good, I do believe code examples are useful at all levels, i.e. class, function, event, property , etc because if nothing else, they make the documenter consider how much their docs make sense. One criticism is that there still tends to be an element of documenting the obvious, i.e. set context, ‘sets the context’.

    I think your best bet is to use analytics and in additioni add insentive for feedback at a post experience point later in the day or a few days later. Maybe you could find a way to track user’s usage and then present it back to them as a report which they could comment on, maybe you could have some kind of log/history that I could refer to.

    On the whole, I think you guys are doing a good job of the docs and the very fact that you/Adobe are asking the question is good in itself.

  17. Pingback: Reasons why visitors might not leave page ratings « Adobe Learning Research Blog