For some time I have been a little concerned that we are making life too simple for ourselves when mapping the different layers of the ICOPER reference model to the relevant standards. We should also observe that there are different “families” of standards, and allow for a discussion on what family is the best match for the competency domain.

In Simon Grants excellent blog posting on our last deliverable with the long title “Model for describing learning needs and learning opportunities taking context ontology modelling into account” there are some seeds for the discussion I call for.

Then it [the deliverable] goes on to suggests an information model for “Learning Outcome Definitions”. This is a tricky one, as one cannot really avoid IMS RDCEO and IEEE RCD. As I’ve argued in the past, I don’t think these are really substantially more helpful than just using Dublin Core, and in a way the ICOPER work here implicitly recognises this, in that even though they still doff a cap to those two specs, most of RDCEO is “profiled” away, and instead a “knowledge / skill / competence” category is added, to square with the concepts as described in the EQF.

When Simon is referring to Dublin Core, it is the DC Abstract model he thinks of, the model based on RDF and semantic web. I wonder if the specifications we turn out as a result of the ICOPER project is “web architecture enabled” – or if our focus is still the repository view of walled “content spaces”, be it OICS or others.

Technorati Tags: icoper, nordlet, SC36

Scott put a screenshot of the ICOPER diagram on the Moodle server, added some Wookie magic and voila, there you had the opportunity to move a number of standard & specification labels around to find where they fit in the big picture. It is best done collaboratively in a small group where you can see where the labels are moving - and you may have a chat or voice channel to help you negotiate what is the best fit.

What a nice way to survive a standards meeting!

Try it yourself at the Univesity of Bolton VLE

Technorati Tags: icoper

If you want to participate, please vote for date and time for the first workshop

Please inform others in you community that will be interested in contributing to the success of this standard.

You will find more information on the workshops at http://wiki.teria.no/confluence/display/mlr5/Developing+MLR+Part+5+Educational

Technorati Tags: Dublin Core, metadata, MLR

Technorati Tags: icoper, widgets, Web2.0

What more to be said? Well, pity on the colleagues that don’t see the value in microblogging.

Technorati Tags: icoper, Web2.0

This is not a review of the paper, just some comments to help my own reflection on the role of Learning, Education and Training standards. Erik & Katrien want to promote “appropriate standards development”, and they want to address misunderstandings regarding the relationship between standards and research, i.e., they discuss the development life cycle of standards. So, what concept of standards do E&K have?

The reason I ask this question is a nagging feeling that we talk about different things all the time we talk about standards being good for this or that, being different from specifications and brought to market with vanilla or chocolate tastes, dependent upon their origin in formal standards fora or just communities of interest. Paul Hollins and I have pointed to the need for a domain theory for learning technology standardisation. What kinds of standards, from which standard setting bodies (SSB), addressing what part of the LET domain – often this is too muddled to have a good conversations, and we see that the discourse get divided along some tragically old axis, West against Central; North against South and so on.

You can enter a discussion on standards through definitions or examples. E&K talk about “accredited standards”, and “accredited standards bodies [as the ones that] define actual standards”. The defining characteristic of this very interesting concept of an actual standard is according to E&K that they “carry substantial legal weight”. To me, it seems that the authors enter the discussion on standards governance without really trying to define their concepts, relying on that the pure weight of The Mothers of All Learning Technology Standards, the IEEE LOM and ADL SCORM, will do the trick of explaining what it is all about.

True, E&K mention a number of other standards being developed through the last decade, but there is no doubt that LOM and SCORM is their index terms to use for thinking about LET standards. To me, two annoying experiences spring to mind. First, the years of long opposition to create a standard on Metadata for Learning Opportunities that was not an application profile of LOM. (Yes, from a nail’s perspective the only tool you need is a hammer.) Second, during the launch of LETSI as the steward of SCORM, the military men i civil suits had the guts to sell SCORM in any current and future version as the ultimate solution to all learning needs in a way that made all teachers look like tank soldiers training to meet Iraqi road bombs.

And my point? being that we have to be aware of the context of the standards we are discussing. And I am not so concerned any longer about formal and ritual contexts of standards, e.g., which SSB accredited the standard. I am more concerned about how a standard enables “an open global learning infrastructure”, as E&K put up as their vision.

I share their vision. However, I will again question the theoretical foundation the vision builds on. E&K’s basic building blocks in such an infrastructure resort to their two mother standards perspective with an add-on of attention metadata (SCORM 2.0?): What it is all about is finding and deploying learning resources. And “in the periphery” we should find some value adding experiencing using mash-up technologies…

Misguided
Erik Duval and Katrien Verbert find the ongoing TEL standardisation “somewhat misguided”. At Erik’s blog he summarizes their main points as follows:
1. The main issue is no longer that we do not have sufficient standards. Rather, we have maybe too many and, more importantly, we don’t make use of them in very advanced ways… Tools are lacking or too much let the standard shine through, rather than focusing on the user experience.
2. We should avoid continuing the ‘not invented here’ approach that has made us develop learning specific standards when there may be quite appropriate standards already out there or being developed.
3. Standards should not be research oriented but rely on proven practice. Of course, standards enable deployment at large scale, and therefor make it possible to do research on global infrastructures.
4. Standards enable openness, and that enables innovation - that is another way for standards to be relevant to research.

Let’s take one point at a time. E&K have a very good point in warning against “preemptive” standardisation, not building on proper needs and analysis of the needs of the end-users. It is true that standardisation take a long time, and there is a danger that projects may “run out of steam”. The order of things should not be, first a standard, then the tools. Quite opposite, we should have a solid practice to build our standards on, and yes, preferably with more than two actual working implementations before we freeze the technology in the form of a standard.

We have a consensus then? Well, the problem with this kind of high level positions is that we don’t really know what E&K mean by a standard. We know what they mean by an accredited standard, but when they talk about standardisation the beast is somewhat hairy. For instance, what do exactly the authors mean when saying “in our experience, rather than developing an alternative specification like Common Cartridge or SCORM2.0, the most urgent and relevant work with respect to learning content is the development of more useful and usable authoring tools, delivery platforms, management systems, etc”?

Alternative to what? E&K don’t tell, but we could guess. Instead of new standards they prefer application profiles and reference models, as “nice vehicles” to build standards on existing ones. However, In the case of Common Cartridge it is a reference model that IMS has built. Concerning SCORM2.0 nobody knows what this will be, as LETSI is still in the process of gathering use cases.

I don’t get the point why it was wrong to develop Common Cartridge (because we already have SCORM or because we don’t yet have any tools using CC?). So let’s see what the authors mean my avoiding “not invented here”. One very interesting example they use is the Simple Query Interface (SQI), developed within CEN/ISSS WS-LT. “In retrospect, we believe that SQI could probably have built on SRU/SRW more directly.” Well, the question was raised many times during the development of SQI, so it is a well founded self-criticism Erik does here.

SQI was clearly a project that set out to develop an alternative to technologies that were already there. It is a question if this should have been done within the context of pre-standardisation, i.e., within CEN/ISSS WS-LT. It depends on what role we give pre-standardisaton, an issue E&K do not discuss as they seem to put all standardisation activities into one big box.

Common Cartridge on the other hand, seems to come out of well defined needs defined by the publishing business, building on existing specifications and standards, e.g., LOM, IMS CP, IMS QTI, IMS Tools Interoperability. To say it was wrong to develop the CC specification seems to overlook the pedagogical, political, and organisational context of this project. In the aftermath of the bitter struggle between IMS and ADL in SC36 and other fora, it is easy to see that CC is developed to break the monopoly of SCORM, at least as a reference point to learning technology standards for schools and higher education. So you have to see the standardisation projects in the context of real life struggle for political influence – for vendors, researchers, institutions, and proponents of specific pedagogical views. Doing so, it seems a bit of wishful thinking to say that certain projects should not be started?

“Standards are not research”
E&K caution against entering the standardisation process prematurely. “The focus of that process is on consensus building, not on developing the best solution.” They therefore propose as a “rule of thumb (…) that a specification is only ready for the standardization process once it has been implemented by at least two independent development teams, and has been evaluated in at least two independent user studies”. These are good rules of thumb, already in use by organisations like IMS and W3C. But again, I wonder about E&K’s concept of standardisation. You have on the one side specification development; on the other you have standardisation, which is purely about consensus building?

Let’s see how they explain the difference between standardisation and research. E&K present a diagram. Research produces specifications that are fed into standardisation. Development on the other hand, implements specifications and produces software. Standardisation learns from software and produces standards. And Use in practice uses standards and feeds into evaluation used by research.

The bottom line of this figure is that specifications and standards are very different things, the former being part of the research sphere, while standards are part of the standardisation sphere. Which brings us back to the question already asked: how do you really define a standard and what makes it different from a specification. E&K use consensus building focus as a defining criterion for a standard, the idea being then that a specification is all about putting “the right stuff” into the design document?

Need for a better standardisation theory
I share Erik Duval and Katrien Verbert’s concern about the need to promote more appropriate standards development. Furthermore, I fully share their views on the need to build standards on real and well documented needs of the end-users. We should ensure that practices are harnessed in working tools before we codify these practices in standards with a long life and serious implications, e.g., for public procurement procedures. However, we need a better understanding of the role specifications and standards play in the development of learning technologies.

In my opinion the ultimate goal of standardisation in our domain is to facilitate good conversations in the design and use of learning technologies, taking place at a number of “conversation layers”. (We could for example, use the European eGovernment Interoperability Framework model and talk about political, legal, organisational, semantic, and technical interoperability levels.) If we stay with this metaphor, every proposal for a change in the way we do things is another way to say: We what to hear your voice stronger, and we want you to be more quiet. In E&K’s case, I gather they want the voice of research to be stronger, the same with software developers; while standards should be “the infrastructures that work behind the scenes”, more quietly. I see no problem with this on a more general level. Less so, if we succeed in applying openness as the driver for innovation, as E&K suggest. However, if we quiet the voice of standardisation, we will on a immediate and practical level be stuck with the standards we have. I agree with E&K that “it is important to make sure that standards enable, and do not prevent pedagogical innovation”, but I do not trust LOM and SCORM to be the prime proponents of such an innovation.

So, what is my proposals towards the aim of appropriateness in standards development? I think we need to rethink the standards development and adoption life cycle within the LET domain and, according to the recommendations of E&K, tie the process better to the needs of the end-users. Therefore, a few points as suggestions to continue the very useful discussion Erik and Katrien have initiated:

1. It does not help the discussion on LET standardisation to try to keep specification from standards; accredited standards from community specifications, and so on. We need to be aware of the potential harmful effects of a inferior specification being turned into for example a European Norm with legal implications. But the main differences between specifications and standards seem to be about branding and marketing, ref. the run for an ISO stamp of the ADL and IMS communities.
2. We should be careful if we want to mandate certain standards for TEL. Standards catalogue initiatives should be challenged to come up with trustworthy systems for revisions and updating before this kind of catalogues are introduced to the LET domain.
3. We should improve the discourse about TEL standards, i.e., we have to build implementors’ communities that have a lively and continuos discussion about the pedagogical and end-user perspectives on the technologies that use standards. Even if standards work behind the scenes, they are not value neutral. Only a good discourse will enable us to choose the right standard for the right task, and to help us see where to look when we have new needs for solutions.
4. We should spend more time on developing good methodologies for standardisation. By methodologies I also mean sort of guidelines for making the right standards. It goes without saying that it is pretty stupid to introduce huge information models to a field that is very much in a flux, ripe with emerging technologies and practices. But that is what we have been doing.
5. We should prioritise pre-standardisation more, and ensure that formal standardisation, e.g., in ISO/IEC JTC1 SC36 and CEN TC 353, does not embark on projects that have not a well defined scope and are not founded on documented needs. We have examples at least in SC36 that this is not always the case.
6. We should be more observant of the development of community standards, often coming out of research and development projects, – we, being the standards community. These standards do their work, often without the contributions of the standards community . However, standardisation is also an important and efficient dissemination vehicle for learning technologies. Which means that to limit the scope of standardisation only to consensus building is too narrow.

Technorati Tags: Technology standards

The WG3 in the ISO sub committee 36 has just released a draft for a Conceptual Reference Model for Competencies and Related Objects, which is available at the ISO site as PDF. The model is outlined in the figure below:

Comparing the documents, the draft technical report from ISO and the draft reference model from ICOPER, a number of questions arise:

• The information model in the ISO document is quite understandable. The ICOPER document does not outline any model that allows us to see the relations between the main concepts, at least not in a summarising diagram.

• There is no distinction between competency and competence in the ISO document. The ISO document summarises “competency representations can be found in “Competency Information Islands” under the following labels: 1) “Competency” or “Competence” , 2) “Training goals”, “Learning Objectives”, “Educational Objectives”, “Abilities”, Capabilities”; 3) “Aptitude” “. When I suggest that the distinction between competency and competences is a little difficult to grasp, especially by an international audience, I get a pointed finger saying that this difference was explained to me, and that I should go to a web page “that explains perfectly the two terms”. Well, I have read the web page, and I found a tour de force through history of thought since Plato. So I still ask the question: Are the concepts in the ICOPER draft well considered, given the international context of the proposed work?
• The ISO document will be turned into a technical report, that might or might not be a foundation to build applications. As a conceptual reference model it might be looked upon as first stab at the challenge to steer the discussion that might lead to more implementable standards. However, it is interesting to see how the question of who should be leading who is played out in the two documents. Well, indirectly anyway in the case of the ISO draft: ISO is the top of the standards pyramid, and if they try to create global consensus on the top level concepts, that’s what they try to do… The draft from the European ICOPER project shows a figure that could be interpreted in many ways:

In my first blog posting I read this as message to standards fora like CEN and ISO to lay off, waiting for the joint HR-XML IMS initiative to come up with a proposal. In the reply the HR-XML chair says this i incorrect. On the contrary, input from CEN and ISO is most welcome, “because the last thing we need is 5 different standards (IMS, HR-XML, IEEE, ISO, CEN, …) or separate standards for HigherEd, work, K12, etc…” It seems that I have got the figure on the standards evolution wrong then. Nevertheless, my question of who will be leading who is till open. Is the ISO proposal for a conceptual model a good starting point for the work on competency data management? If not, how should we try to influence the report? If it is good work, how will that influence the ICOPER work, that to my surprise “will develop the IEEE Competency Profle (CP), in a way that enables the representation of competencies (and relations between them) from both academic and workplace competencies” (page 6)?

(Do I need to state that the opinions in this blog post are mine alone, and that no organisation, neither Norwegian nor European, is to blame for my points of view. And that they have nothing to do with my role as project partner in any project. No, but said anyway.)

Technorati Tags: competencies, icoper, IEC JTC1 SC36

At the present stage a study of JISC-CETIS’s Phil Baker has been of great help. The Learning Material Application Profile Scoping Study is available in draft from Phil’s blog. My proposal for the next editors meeting is to use the model Phil comes up with a starting point to discuss the top level concepts of MLR Educational.

First, some insights found in Phil’s report. He points out that metadata for education is one of the domains where the issues are least well articulated and where solutions are least developed. His study shows that where resource descriptions were seen as “difficult” there seems to be suggestions to move away from structured metadata towards the approach of providing semi-structured free text descriptions. This should be seen as a warning signal to SC36 colleagues who want to be too explicit in modelling educational metadata. We are making an international standard that should cover the needs of all educational levels in all regions and cultures throughout the world, a truly daunting task!

Phil is of the opinion that any future work should recognise that metadata may be valuable in supporting resource management, discovery/retrieval and use. And he reminds us that the application profile to support resource discovery might be quite different from an application profile to support resource management or use.

Therefore, we should come up with use cases that show a number of scenarios:

• The Provider role
• The Management role
• The Consumer role

The last role should not just cover the users of repositories in a strict sense, but also reflect the emergent use of learning resources through participation in different Communities of Practice, in digital living and serendipitous learning, i.e., the discovery use case.

Furthermore, Phil has a concept of the Primary Resource, the Learning Material; and the Secondary Entities that are related to the learning material and which have properties that need to be specified in order to provide full description of the primary resource. This is similar to my concepts of intrinsic and extrinsic properties of a learning resource. However, Phil uses the FRBR model to unpack the intrinsic qualities of the learning resource.

I have copied the model Phil Baker has made to illustrate what is described by metadata for learning materials. Whether we in MLR Part 5 should bring into play the full Functional Requirement for Bibliographic Records (FRBR) entity model is to be discussed. I would think that this will vary according to the needs of different communities, and that in some application profiles you will find the full model, and in others a reduced set.

What we should discuss, is this simple model, defining a learning resource by Subject, Audience, Agent and Educational Context.

We should list the metadata elements that the different National Bodies have launched as candidates for the new standard, and see if we are able to place them as refinements of these top level concepts, e.g., audience language, educational level, accessibility preferences, pedagogical modality, competency or learning outcome, etc.

The tricky bit will be to tackle the redundancies in this model, e.g., is educational level a property of Audience or Educational Context. Or is Audience in fact a property of Educational Context? I bet this discussion will show that this depends on the use scenarios, and that we will not be able to come up with a model that is completely streamlined at this stage of learning technology development.

Technorati Tags: metadata, MLR, SC36

If you are interested in the European work on competency frameworks and standardisation issues concerning competencies I would recommend you go to to the Symposium wiki to find links to the recordings, papers and presentations. Also, I would recommend to have a look at the video clips from the discussions at icoper.blip.tv.

The symposium results will be reported to CEN/ISSS WS-LT at the next meeting in Athens 9 January, and I will therefore point to the draft of the proposed reference model as a document that needs to be discussed, both in the ICOPER project and within the standards community.

The document is based on some earlier work by Luk Vervenne, chair of HR-XML, late Claude Ostyne of IEEE LTSC WG20 and now Jehad Najjar, who works in the ICOPER project. It is an early draft, and it is not clear if this could be presented as the consensus of the Berlin symposium or the ICOPER project. Anyway, it is a good starting point for the broad discussions that is needed to harmonise the views of the entreprise world and the HE community on competency issues.

The scope of the reference model is to meet “specific business requirements identified in application areas such as education, personal development and learning content development as well as hiring, performance improvement and training”. In other words, this initiative tries to use the technologies developed in the training and enterprise sector (which already has an established market for competency technologies and services) to embrace higher education and life long learning, encouraged by the EU policies to implement a European Qualification Framework though National Qualification Frameworks. The proposal points to the HR-XML and IMS joint effort to create “Common Semantics for Employability” as the vehicle to achieve this, and indirectly asks other standards fora to lay off, waiting for the process to be finalized in ISO/IEC JTC1 SC36.

In my opinion the draft calls for a broad discussion across the different European projects represented in the Berlin Symposium, and within European standards groups. The reason for this is that I am not at all convinced that we have a consensus on the proposed reference model.

The Concepts
The draft comes up with a number terms and definitions. And it seems that some definitions are lacking, others are presupposing an abstract model that is not outlined in the draft. Competency is said to have three subcomponents, knowledge, skills and abilities. The last concept is not defined in the draft. Knowledge is in fact defined as information processing (”assimilation of information through learning”), which is highly contested. The definition of learning outcomes introduces a concept of attitudes, not defined in the model. The concept of Competency Framework is equivalent to a repository, which differs from the way the term Framework is used in many standards. Furthermore, the model uses the terms competency and competence (the latter being competency in a particular context). The discussions so far in international fora shows that this distinction is very hard to explain to non English communities, and it make translations to other languages very difficult.

It seems that the authors behind the draft have got tired of trying to define their concepts, as they end the section on definitions with the the following: “Nevertheless, in this reference model it is possible to collect and manipulate data relating to competency, regardless of what competency means.” You may fish with this approach, but if you catch a shark and not only fish, you should be OK. Well, maybe?!

A model for management
The draft outlines a model for Competency Data Management. I would ask if this is not a too narrow perspective for a model addressing the needs of a European Competency Framework. Especially, if this is harnessed in a repository only, ref. the definition of a Competency Framework in the draft. Added the large data model and XML “bias” of the draft, I wonder if the approach taken is more adapted to the needs of the 80’s and 90’s than the future knowledge society. I have the feeling that the sheer complexity of the proposed reference model is going to be a major barrier to implementation.

Cultural issues
The last issue I will raise addresses the model’s ability to engage higher education, the prime target group of the ICOPER project. In fact, a reference model should be a “template” for discussion with the communities interested in implementing the standards and specifications that are referenced. My concern is that the scenarios for building for example competency profiles should use Higher Ed use cases, capturing the full range of HE activities, not only training for “high risk professions” (pilots, drivers, medical doctors, etc.) The case of baking bread (often used in discussing these issues) might be illustrative for training, but how relevant is it for what happens in higher education?

Technorati Tags: competencies, icoper, metadata