JEFF THOMAS COBB

Leading with Learning

learning management system

Association Learning Management System Report Published

By Leave a Comment

Association LMS coverI talk a fair amount about creating and marketing digital products here on Hedgehog & Fox. I don’t do this purely from a theoretical or consultative perspective: digital products in the form of research reports are actually an important part of my own business over at Tagoras. So, I am happy to announce the publication of our newest report: Association Learning Management Systems.

You can get the full scoop on the  Association Learning Management Systems report page on the Tagoras site, but I’ve included some of the key details below for readers here who may be interested.

Whether of not you want the full report, I recommend downloading the table of contents and sector overview. It’s got some good content in it. Here are a few of the key details on what the full report – which is more than 400 pages long – includes:

  • An overview of LMS usage in the association sector
  • An extensive set of guiding questions for LMS selection
  • 29 tables that compare systems across a variety of areas
  • In-depth vendor profiles based on 243 questions covering 28 different areas
  • Brief reviews of each system based on answers to the questions and a demonstration session with each vendor

Companies included in the report are:

      In case it is not immediately obvious, a report like this can save you a tremendous amount of time and money if you are planning to run a learning management system RFP selection process.

      Jeff

      Update, March 2010: Individual LMS profiles from the Association Learning Management System report have now been released. See the Tagoras association learning management system profiles press release

      LMS Selection Resource

      By Leave a Comment

      My sense from numerous conversations at conferences like the recent ASAE e-learning conference is that the process of identifying and selecting a learning management system (LMS) remains shrouded in mystery for potential licensees. Tony Karrer has been addressing LMS selection on his blog for some time and recently capped off a number of postings that Mission to Learn readers may find helpful with the following slides from a DevLearn presentation.

      A couple of things to highlight in the slides:

      • I like the “Featuritis Curve” from Kathy Sierra that Karrer makes use of in the slides. As Sierra illustrates, adding features is valuable up to the point that a “Happy User Peak” is reached. After that it’s all down hill.

      In my own experience, the average LMS selection process becomes focused very quickly on features. This is partly natural and understandable: features tend to be the way in which a user “knows” a software product, so they represent an easy and obvious area on which to focus.

      But this natural tendency is also fed by a dysfunctional “chicken or egg” dance between developers and users in which the introduction of new features, whether requested by the user or simply added because the developer thinks it will create competitive advantage, leads to a desire for yet more features. As I have commented before, and most recently on Ellen Behren’s blog, I think the average RFP process contributes to this dynamic.

      The second thing I’d like to highlight from Karrer’s slides is his top three common selection mistakes:

      3.    Not having differentiating use cases
      2.    Customization
      1.    Putting the wrong requirements in an RFP

      In my mind, these three are very closely related, and I would actually put Karrer’s #3 at #1 because I think it leads to the other two. I have commented before on the importance of use cases. The fact that they are done poorly or not at all is one factor, in my experience, that drives both customers and vendors to feel that customization is needed when in fact it is not. And this, of course, is one of the factors that leads to wrong requirements in an RFP.

      Karrer seems to assume that an RFP is an inevitable part of the process. As I’ve indicated in earlier postings, I disagree. I have perhaps become overly cynical, but I feel the average RFP process (at least in the association sector, where I currently do most of my work) is a boon for consultants and a crutch for organizations. Nonetheless, I realize they are not going away anytime soon, and I appreciate the fact that Karrer’s slides wrap up with a distinction between bad RFP requirements and good RFP requirements.

      In general, there are good thoughts in this presentation and elsewhere in Karrer’s writing for those readers considering their first (or second, or third…) LMS.

      JTC

      P.S. Not also that Karrer is one of the speakers at the upcoming (and free) corporate e-learning conference.


      I’M IN UR LMS

      By Leave a Comment

      I came across this funny and instructive little Flash movie on Cathy Moore’s Making  change: Ideas for lively elearning blog this morning (a great site for anyone even remotely interested in online learning to visit) and once again could not resist a Friday posting. For Mac users, it’s worth noting that this was created using Keynote. Have fun, and TGIF.

      JTC


      A Modest Proposal: Kill the RFP, Part II

      By 2 Comments

      In this posting I continue the musings on the efficacy of Requests for Proposals (RFPs) that I began in “A Modest Proposal: Kill the RFP, Part I”

      I rarely see an RFP in which an organization has truly, in a disciplined, measurable fashion, mapped objectives for an online learning technology initiative back to overall strategic objectives for the organization. (And, for what it is worth, I have never seen e-learning features analyzed in this way in an RFP.) This, in my opinion, is information that should be provided to a vendor, and the purchasing organization should expect vendors to be able to engage in meaningful dialogue around these areas. “Dialogue” is the key term, however. I can’t think of an e-learning RFP I have received in recently that asked for the vendor’s reactions to the organization’s proposed objectives for the implementation of online learning technologies.

      Indeed, whatever information the organization offers is generally presented as set in stone, and the vendor is simply supposed to provide features that purportedly support the organization’s online learning strategy.

      In Exceptional Selling: How the Best Connect and Win in High Stakes Sales, Jeff Thull, who I referenced in Part I of this posting, describes this dynamic as a sort of dysfunctional parent-child relationship:

      The request for proposal (RFP)…is an institutionalized version of a parent-child or superior-subordinate transaction. The customer sends out a parental command to bid a project, the “to-do” list, and many sellers automatically obey, often investing huge amounts of time, money, and other resources in the proposal process without question.

      It is—or at least should be—the case, of course, that an organization will know its own online learning technology strategy better than any vendor that is hired to perform services for a specific initiative. Nonetheless, a sort of Heisenberg uncertainty principle applies—as the organization and the vendor consider the strategy together, in the light of the additional knowledge and experience the vendor brings to the table, the strategy is bound to change. A good vendor—one that has the potential to become the proverbial “partner”—should be
      very comfortable engaging in this process and should bring a lot of value to it.

      How do you find that vendor? To be honest, the best of them are going to be likely to turn down the opportunity to bid on any RFP you send them unless they have been instrumental in crafting the RFP. Once you have brought the right stakeholders together internally to articulate clear, measurable objectives for your e-learning initiative, I
      recommend developing a list of no more than five potential candidates based on a combination of Web-research, recommendations from peers who have already launched online learning initiatives (consider posting to the DC Distance Learning Coalition list, for instance), and if necessary, a brief request for information from prospective online learning technology vendors.

      Next, share your online learning technology strategy with the vendors and schedule a 60 to 90 minute call with each to discuss how they can work with you most effectively to achieve your objectives. When these conversations happen, pay attention to the types of questions the vendor representatives ask and how well they listen. Chances are you will
      be able to tell pretty quickly which ones are capable of engaging in meaningful dialogue about e-learning technology strategy—driven primarily by listening to and asking questions about your objectives—and which ones are prepared only to extol the features of their products.

      After the initial phone conversations, narrow the list and meet with the finalists in person. I recommend dividing this over a two day period. On the first day, focus on demonstration of the vendor’s offering. In as much detail as possible—using texts, diagrams, examples from other online learning initiatives you like—explain to the vendor ahead of time what you think the most common things a typical end user and the typical administrator at your organization will need to be able to do to make your online learning initiative a success. Ask the vendor to walk through these scenarios in detail, explain the functionalities the vendor’s product can offer to support each scenario, and suggest how any functionality gaps will be addressed.

      On the second day—preferably in a different room, perhaps even over lunch—spend more time discussing your objectives and also probing on online learning technology implementations the vendor has done before with similar organizations.

      Finally, only once all of the above is accomplished, it is valuable to ask the vendor to which you believe you will award the work to submit, in writing, a document that briefly outlines the vendors understanding of your e-learning business objectives, states what learning technology products and services the vendor will provide to meet those objectives, and indicates the pricing and timeline involved. At this point, the vendor should be very well prepared to provide this information in a meaningful way and you should be feeling good that you know what you are looking for. This “proposal” document should therefore be of dramatically higher quality that what you would receive in the typical RFP process.

      To close, I know many organizations will object that their bylaws require them to issue RFPs. In my experience, the real requirement is usually for a competitive bidding process, which does not necessitate an RFP. If an RFP truly is required, I think the above steps could actually be structured to function as a more formal process. But if at all possible, I
      advise organizations to stick by my original recommendation: Kill the RFP.

      Jeff

      P.S. – If you like what you read here on Hedgehog & Fox, in general, I’d truly appreciate it if you would tell others and also subscribe by RSS or by e-mail.


      A Modest Proposal: Kill the RFP, Part I

      By Leave a Comment

      [Note: This is a two part series. I wrote it back when I was a learning technology vendor on the receiving end of many RFPs. I am no longer a vendor, and only in very rare instances will I agree to respond to RFPs.]

      I’ve recently been reading Exceptional Selling: How the Best Connect and Win in High Stakes Sales, the latest from my favorite writer on sales strategy, Jeff Thull, and while Thull does not write specifically about e-learning or online learning technologies, doing so has reinvigorated a conviction that I have held for some time: the bulk of RFPs issued for the purchase of online learning technologies do at least as much harm as good. Indeed, I believe this is likely true for nearly all product and service purchases, but I’ll do my readers the favor of sticking to territory I know something about.

      You see, I receive many well-intentioned RFPs throughout the course of any given year, all of them aimed at securing services for the implementation of online learning technology initiatives—primarily license of a learning management system (LMS) or development of online courses. These documents inevitably contain a very long list of features the prospective purchaser is seeking, as well as frequent use of well-worn terms like “interactive” and “robust.” The better of them also contain at least some information about the purchaser’s e-learning business objectives and measures for success. A rare few contain uses cases (i.e., scenarios that map out in great detail the steps end users and administrators take to use a learning technology and derive value from it) that relevant stakeholders within the organization have attempted to articulate. I plan to address the development of quality use cases in a future posting, but let’s go ahead and consider feature lists and business objectives.

      Feature lists, I will admit, are probably at the root of my distaste for online learning technology RFPs. Partly that is because responding to them in a thoughtful manner takes significantly more time than I suspect most RFP issuers realize. More importantly, I think they are ultimately of very limited use. I understand the impulse to include them, of course—and I fully realize that the emphasis many e-learning vendors place on features makes these lists almost inevitable—but too often they serve as a substitute for a truly responsible, strategic decision-making process. The organizational stakeholders responsible for issuing the RFP can point to a list as evidence of their diligence; consultants can play their role by generating the lists and processing the responses; and sales people can feel they have done their jobs by having responded to the lists. Superficially, the needs of all parties are satisfied.

      There is any number of problems with this approach, but for the time being I’ll just highlight two. The first is that features, as they appear on paper, are always open to interpretation. What one e-learning vendor means by “Assign students to groups or departments” or “Built-in Authoring Tool,” for example, can actually vary pretty dramatically. Arguably, these differences come out during demonstrations, but in my experience, most organizations—and, surprisingly, a large percentage of consultants—either don’t know the right questions to ask to understand these difference, or neglect to ask them. The latter may happen so often simply because the list of features that the vendor might cover in a demonstration is so long. (I have received RFPs for learning management system purchase with lists of features more than 12 pages long!)

      The second problem I have with feature lists is that, if an organization really wants a feature-by-feature comparison, there are already good options for getting this information without going through an RFP. For starters, these can be compiled from most vendors’ Web sites or received from the vendor as part of a simple request for information. If the organization does not want to go through this effort—though, in my opinion, most would benefit from insight it offers—it is often possible to purchase reports that detail the range of features various vendor products offer. Brandon Hall Research and Bersin & Associates both offer an excellent range of reports on various online learning technologies and services, for instance. And while the price tag for these can be somewhat steep, they still are likely to cost less than the many hours of staff or consultant time organizations invest in for compiling lists. [Note: Since writing this post, I have launched a research company that recently issued a report on LMSes for Associations: Association Learning Management Systems.]

      Those hours that might be spent by internal staff and/or consultants to develop feature lists could be much more fruitfully applied to developing e-learning business objectives—a topic I will focus on in my next posting about RFPs.

      Jeff

      P.S. – If you like what you read here on Hedgehog & Fox, in general, I’d truly appreciate it if you would tell others and also subscribe by RSS or by e-mail.