eBlog by Tore Hoel

Erik Duval is practising open research in a fine way that fosters a much needed discussion on the role of technical standards for learning technologies. At his blog he shares the version of an article he and his colleague Katrien Verbert have submitted for review. I was sleeping and missed the chance to influence the paper; I’m glad we have the chance to use it as at starting point for a discussion on how we are going to improve standards activities for technology enhanced learning (TEL).

This is not a review of the paper, just some comments to help my own reflection on the role of Learning, Education and Training standards. Erik & Katrien want to promote “appropriate standards development”, and they want to address misunderstandings regarding the relationship between standards and research, i.e., they discuss the development life cycle of standards. So, what concept of standards do E&K have?

The reason I ask this question is a nagging feeling that we talk about different things all the time we talk about standards being good for this or that, being different from specifications and brought to market with vanilla or chocolate tastes, dependent upon their origin in formal standards fora or just communities of interest. Paul Hollins and I have pointed to the need for a domain theory for learning technology standardisation. What kinds of standards, from which standard setting bodies (SSB), addressing what part of the LET domain – often this is too muddled to have a good conversations, and we see that the discourse get divided along some tragically old axis, West against Central; North against South and so on.

You can enter a discussion on standards through definitions or examples. E&K talk about “accredited standards”, and “accredited standards bodies [as the ones that] define actual standards”. The defining characteristic of this very interesting concept of an actual standard is according to E&K that they “carry substantial legal weight”. To me, it seems that the authors enter the discussion on standards governance without really trying to define their concepts, relying on that the pure weight of The Mothers of All Learning Technology Standards, the IEEE LOM and ADL SCORM, will do the trick of explaining what it is all about.

True, E&K mention a number of other standards being developed through the last decade, but there is no doubt that LOM and SCORM is their index terms to use for thinking about LET standards. To me, two annoying experiences spring to mind. First, the years of long opposition to create a standard on Metadata for Learning Opportunities that was not an application profile of LOM. (Yes, from a nail’s perspective the only tool you need is a hammer.) Second, during the launch of LETSI as the steward of SCORM, the military men i civil suits had the guts to sell SCORM in any current and future version as the ultimate solution to all learning needs in a way that made all teachers look like tank soldiers training to meet Iraqi road bombs.

And my point? being that we have to be aware of the context of the standards we are discussing. And I am not so concerned any longer about formal and ritual contexts of standards, e.g., which SSB accredited the standard. I am more concerned about how a standard enables “an open global learning infrastructure”, as E&K put up as their vision.

I share their vision. However, I will again question the theoretical foundation the vision builds on. E&K’s basic building blocks in such an infrastructure resort to their two mother standards perspective with an add-on of attention metadata (SCORM 2.0?): What it is all about is finding and deploying learning resources. And “in the periphery” we should find some value adding experiencing using mash-up technologies…

Misguided
Erik Duval and Katrien Verbert find the ongoing TEL standardisation “somewhat misguided”. At Erik’s blog he summarizes their main points as follows:
1. The main issue is no longer that we do not have sufficient standards. Rather, we have maybe too many and, more importantly, we don’t make use of them in very advanced ways… Tools are lacking or too much let the standard shine through, rather than focusing on the user experience.
2. We should avoid continuing the ‘not invented here’ approach that has made us develop learning specific standards when there may be quite appropriate standards already out there or being developed.
3. Standards should not be research oriented but rely on proven practice. Of course, standards enable deployment at large scale, and therefor make it possible to do research on global infrastructures.
4. Standards enable openness, and that enables innovation - that is another way for standards to be relevant to research.

Let’s take one point at a time. E&K have a very good point in warning against “preemptive” standardisation, not building on proper needs and analysis of the needs of the end-users. It is true that standardisation take a long time, and there is a danger that projects may “run out of steam”. The order of things should not be, first a standard, then the tools. Quite opposite, we should have a solid practice to build our standards on, and yes, preferably with more than two actual working implementations before we freeze the technology in the form of a standard.

We have a consensus then? Well, the problem with this kind of high level positions is that we don’t really know what E&K mean by a standard. We know what they mean by an accredited standard, but when they talk about standardisation the beast is somewhat hairy. For instance, what do exactly the authors mean when saying “in our experience, rather than developing an alternative specification like Common Cartridge or SCORM2.0, the most urgent and relevant work with respect to learning content is the development of more useful and usable authoring tools, delivery platforms, management systems, etc”?

Alternative to what? E&K don’t tell, but we could guess. Instead of new standards they prefer application profiles and reference models, as “nice vehicles” to build standards on existing ones. However, In the case of Common Cartridge it is a reference model that IMS has built. Concerning SCORM2.0 nobody knows what this will be, as LETSI is still in the process of gathering use cases.

I don’t get the point why it was wrong to develop Common Cartridge (because we already have SCORM or because we don’t yet have any tools using CC?). So let’s see what the authors mean my avoiding “not invented here”. One very interesting example they use is the Simple Query Interface (SQI), developed within CEN/ISSS WS-LT. “In retrospect, we believe that SQI could probably have built on SRU/SRW more directly.” Well, the question was raised many times during the development of SQI, so it is a well founded self-criticism Erik does here.

SQI was clearly a project that set out to develop an alternative to technologies that were already there. It is a question if this should have been done within the context of pre-standardisation, i.e., within CEN/ISSS WS-LT. It depends on what role we give pre-standardisaton, an issue E&K do not discuss as they seem to put all standardisation activities into one big box.

Common Cartridge on the other hand, seems to come out of well defined needs defined by the publishing business, building on existing specifications and standards, e.g., LOM, IMS CP, IMS QTI, IMS Tools Interoperability. To say it was wrong to develop the CC specification seems to overlook the pedagogical, political, and organisational context of this project. In the aftermath of the bitter struggle between IMS and ADL in SC36 and other fora, it is easy to see that CC is developed to break the monopoly of SCORM, at least as a reference point to learning technology standards for schools and higher education. So you have to see the standardisation projects in the context of real life struggle for political influence – for vendors, researchers, institutions, and proponents of specific pedagogical views. Doing so, it seems a bit of wishful thinking to say that certain projects should not be started?

“Standards are not research”
E&K caution against entering the standardisation process prematurely. “The focus of that process is on consensus building, not on developing the best solution.” They therefore propose as a “rule of thumb (…) that a specification is only ready for the standardization process once it has been implemented by at least two independent development teams, and has been evaluated in at least two independent user studies”. These are good rules of thumb, already in use by organisations like IMS and W3C. But again, I wonder about E&K’s concept of standardisation. You have on the one side specification development; on the other you have standardisation, which is purely about consensus building?

Let’s see how they explain the difference between standardisation and research. E&K present a diagram. Research produces specifications that are fed into standardisation. Development on the other hand, implements specifications and produces software. Standardisation learns from software and produces standards. And Use in practice uses standards and feeds into evaluation used by research.

The bottom line of this figure is that specifications and standards are very different things, the former being part of the research sphere, while standards are part of the standardisation sphere. Which brings us back to the question already asked: how do you really define a standard and what makes it different from a specification. E&K use consensus building focus as a defining criterion for a standard, the idea being then that a specification is all about putting “the right stuff” into the design document?

Need for a better standardisation theory
I share Erik Duval and Katrien Verbert’s concern about the need to promote more appropriate standards development. Furthermore, I fully share their views on the need to build standards on real and well documented needs of the end-users. We should ensure that practices are harnessed in working tools before we codify these practices in standards with a long life and serious implications, e.g., for public procurement procedures. However, we need a better understanding of the role specifications and standards play in the development of learning technologies.

In my opinion the ultimate goal of standardisation in our domain is to facilitate good conversations in the design and use of learning technologies, taking place at a number of “conversation layers”. (We could for example, use the European eGovernment Interoperability Framework model and talk about political, legal, organisational, semantic, and technical interoperability levels.) If we stay with this metaphor, every proposal for a change in the way we do things is another way to say: We what to hear your voice stronger, and we want you to be more quiet. In E&K’s case, I gather they want the voice of research to be stronger, the same with software developers; while standards should be “the infrastructures that work behind the scenes”, more quietly. I see no problem with this on a more general level. Less so, if we succeed in applying openness as the driver for innovation, as E&K suggest. However, if we quiet the voice of standardisation, we will on a immediate and practical level be stuck with the standards we have. I agree with E&K that “it is important to make sure that standards enable, and do not prevent pedagogical innovation”, but I do not trust LOM and SCORM to be the prime proponents of such an innovation.

So, what is my proposals towards the aim of appropriateness in standards development? I think we need to rethink the standards development and adoption life cycle within the LET domain and, according to the recommendations of E&K, tie the process better to the needs of the end-users. Therefore, a few points as suggestions to continue the very useful discussion Erik and Katrien have initiated:

1. It does not help the discussion on LET standardisation to try to keep specification from standards; accredited standards from community specifications, and so on. We need to be aware of the potential harmful effects of a inferior specification being turned into for example a European Norm with legal implications. But the main differences between specifications and standards seem to be about branding and marketing, ref. the run for an ISO stamp of the ADL and IMS communities.
2. We should be careful if we want to mandate certain standards for TEL. Standards catalogue initiatives should be challenged to come up with trustworthy systems for revisions and updating before this kind of catalogues are introduced to the LET domain.
3. We should improve the discourse about TEL standards, i.e., we have to build implementors’ communities that have a lively and continuos discussion about the pedagogical and end-user perspectives on the technologies that use standards. Even if standards work behind the scenes, they are not value neutral. Only a good discourse will enable us to choose the right standard for the right task, and to help us see where to look when we have new needs for solutions.
4. We should spend more time on developing good methodologies for standardisation. By methodologies I also mean sort of guidelines for making the right standards. It goes without saying that it is pretty stupid to introduce huge information models to a field that is very much in a flux, ripe with emerging technologies and practices. But that is what we have been doing.
5. We should prioritise pre-standardisation more, and ensure that formal standardisation, e.g., in ISO/IEC JTC1 SC36 and CEN TC 353, does not embark on projects that have not a well defined scope and are not founded on documented needs. We have examples at least in SC36 that this is not always the case.
6. We should be more observant of the development of community standards, often coming out of research and development projects, – we, being the standards community. These standards do their work, often without the contributions of the standards community . However, standardisation is also an important and efficient dissemination vehicle for learning technologies. Which means that to limit the scope of standardisation only to consensus building is too narrow.

Technorati Tags: Technology standards