# data[].attributes.headline
ITS can simulate authentic environments. -Robby Robson

# data[].attributes.subject
ITS can simulate authentic environments. -Robby Robson

# data[].attributes.headline
ITS are designed to report data. -Robby Robson

# data[].attributes.subject
ITS are designed to report data. -Robby Robson

# data[].attributes.headline
ITS can alter scenarios (statically/dynamically). -Robby Robson

# data[].attributes.subject
ITS can alter scenarios (statically/dynamically). -Robby Robson

# data[].attributes.headline
Time and expense to create scenarios -including GIFT. -Robby Robson

# data[].attributes.subject
Time and expense to create scenarios -including GIFT. -Robby Robson

# data[].attributes.headline
ITS assess outcomes (data translation is needed). -Robby Robson

# data[].attributes.subject
ITS assess outcomes (data translation is needed). -Robby Robson

# data[].attributes.headline
Authors want to report on their own frameworks. -Robby Robson

# data[].attributes.subject
Authors want to report on their own frameworks. -Robby Robson

# data[].attributes.headline
Tools for linking ITS to competencies. -Robby Robson

# data[].attributes.subject
Tools for linking ITS to competencies. -Robby Robson

# data[].attributes.headline
Creation of predictive models using ITS data. -Robby Robson

# data[].attributes.subject
Creation of predictive models using ITS data. -Robby Robson

# data[].attributes.headline
Tools that enable the framework to scenario pathway. -Robby Robson

# data[].attributes.subject
Tools that enable the framework to scenario pathway. -Robby Robson

# data[].attributes.headline
IoT and other systems can report data (more easily). -Robby Robson

# data[].attributes.subject
IoT and other systems can report data (more easily). -Robby Robson

# data[].attributes.headline
Build a robust competency data pipeline and feedback loop. -Robby Robson

# data[].attributes.subject
Build a robust competency data pipeline and feedback loop. -Robby Robson

# data[].attributes.headline
ITS are viewed as tools for education, not training. -Robby Robson

# data[].attributes.subject
ITS are viewed as tools for education, not training. -Robby Robson

# data[].attributes.headline
Bad predictions could sour the market. -Robby Robson

# data[].attributes.subject
Bad predictions could sour the market. -Robby Robson

# data[].attributes.headline
Higher order/composite constructs compared to KSAs are aligned with job requirements-they are what managers care about. - Patrick Kyllonen

# data[].attributes.subject
Higher order/composite constructs compared to KSAs are aligned with job requirements-they are what managers care about. - Patrick Kyllonen

# data[].attributes.headline
Performance assessments, which naturally pair with competencies (vs. MC, ratings), tend to provide strong validity evidence (e.g., content, criterion). -Patrick Kyllonen

# data[].attributes.subject
Performance assessments, which naturally pair with competencies (vs. MC, ratings), tend to provide strong validity evidence (e.g., content, criterion). -Patrick Kyllonen

# data[].attributes.headline
Performance standards, (performance levels, performance level descriptors) enable clear communication with stakeholders. -Patrick Kyllonen

# data[].attributes.subject
Performance standards, (performance levels, performance level descriptors) enable clear communication with stakeholders. -Patrick Kyllonen

# data[].attributes.headline
Competency scenarios can be idiosyncratic (specific), therefore danger of training irrelevant features rather than the broad competency-must develop many diverse scenarios to avoid specificity (increasing development cost). -Patrick  Kyllonen

# data[].attributes.subject
Competency scenarios can be idiosyncratic (specific), therefore danger of training irrelevant features rather than the broad competency-must develop many diverse scenarios to avoid specificity (increasing development cost). -Patrick  Kyllonen

# data[].attributes.headline
ITS approaches have not yet proven advantageous. -Patrick Kyllonen

# data[].attributes.subject
ITS approaches have not yet proven advantageous. -Patrick Kyllonen

# data[].attributes.headline
Performance assessments are expensive (development, administration time) and may be unreliable due to specificities, or alternatively take a lot of user time to assess. -Patrick Kyllonen

# data[].attributes.subject
Performance assessments are expensive (development, administration time) and may be unreliable due to specificities, or alternatively take a lot of user time to assess. -Patrick Kyllonen

# data[].attributes.headline
Many valued competencies are "soft skills" a relatively untapped training domain. There is a potential first-mover advantage. -Patrick Kyllonen

# data[].attributes.subject
Many valued competencies are "soft skills" a relatively untapped training domain. There is a potential first-mover advantage. -Patrick Kyllonen

# data[].attributes.headline
A first-mover advantage is especially true for ITSs--ITSs have traditionally serviced hard skill domains. -Patrick Kyllonen

# data[].attributes.subject
A first-mover advantage is especially true for ITSs--ITSs have traditionally serviced hard skill domains. -Patrick Kyllonen

# data[].attributes.headline
Student, domain modeling have not yet been attempted in soft skill areas--we're early on the growth curve--lots to be learned yet with benefits to learning science beyond ITS itself. -Patrick Kyllonen

# data[].attributes.subject
Student, domain modeling have not yet been attempted in soft skill areas--we're early on the growth curve--lots to be learned yet with benefits to learning science beyond ITS itself. -Patrick Kyllonen

# data[].attributes.headline
Market consolidation (e.g. Blackboard-Anthology) will drive out competition and innovation; since ITS haven't proven value yet, traditional approaches will reign. -Patrick Kyllonen

# data[].attributes.subject
Market consolidation (e.g. Blackboard-Anthology) will drive out competition and innovation; since ITS haven't proven value yet, traditional approaches will reign. -Patrick Kyllonen

# data[].attributes.headline
Expense of competency-based scenario design will favor adoption of less expensive, traditional approaches. -Patrick Kyllonen

# data[].attributes.subject
Expense of competency-based scenario design will favor adoption of less expensive, traditional approaches. -Patrick Kyllonen

# data[].attributes.body
Implement complex performance measures--Mock interviews, Role plays, Character critiques

# data[].attributes.body
Develop additional measures of soft skills--Situational judgment tests, real-effort tasks, collaborative problem-solving tasks, time-, risk-, and social-preference tasks

# data[].attributes.body
Implement Question and Test Interoperability Version 3.0 (QTI v3) specification (IMS Global, 2020) for storing and exchanging items and tests, deploying item banks

# data[].attributes.body
Implement technology for scoring ill-defined, subjective, and complex tasks

•Video record trainee’s performance (e.g., interview, role play) following a prompt

•Retrievable for scoring using playback (e.g., pause, fast-forward) and annotation and input tools (e.g., for comments, ratings).

# data[].attributes.body
Long-term, implement real-time automated scoring based on stimulus features

# data[].attributes.body
GIFT is being used as portal for simulations and other training systems, but imposes its own paradigm (e.g. evaluating performance at, below, or above expectations). This raises the question of whether it could support several different general approaches and act as a unified portal or middleware, connecting to, say, 90% of real world training and education systems. I can see some market advantages if it can do so.

# data[].attributes.body
Zhao and Watterson (2021) say the following: "For humans to thrive in the age of smart machines, it is essential that they do not compete with machines. Instead, they need to be more human. Being unique and equipped with social-emotional intelligence are distinct human qualities (Zhao 2018b, 2018c) that machines do not have (yet). In an AI world individual creativity, artistry and humanity will be important commodities that distinguish us from each other." Do the soft cognitive skills implicated in these commodities differ from traditionally thought of soft cognitive skills?

# data[].attributes.body
Build a “soft skills” competency training system and get it into the field—determine what works, get user feedback
Widespread data collection will enable big data evaluation approaches—what is the nature of successful performance? What do good performers do? What processing steps do they take?

# data[].attributes.headline
Build Authoring Interfaces to Support Competency Modeling

# data[].attributes.subject
Build Authoring Interfaces to Support Competency Modeling

# data[].attributes.body
I agree with Robby... oh my... I can't believe I said that out loud... but we need to make it easier to develop competency models and translate competencies across different experiences (training, job). An interesting trend is open credential publishing.  These systems are supported as individual wallets that can be shared in different training and educational environments by the learner.  Standardizing interfaces for enable competency model data to be developed, shared, and consumed by other ITSs will go a long way to reducing workload associated with building competency models.

# data[].attributes.headline
The Task of Documenting Required Skills and Knowledge

# data[].attributes.subject
The Task of Documenting Required Skills and Knowledge

# data[].attributes.body
As a basis for generating learning objectives or concepts to learn, the task of documenting skills and knowledge required to do a job on a day to day basis is difficult... massively difficult.  It often requires many SMEs to determine the requirements and then the set of experiences needed to develop need knowledge and master essential skills.  The more we can distribute the workload of this process the more prevalent and accurate competency modeling will be.

A working group to help solve this problem is needed.