SWOT Analysis of ITS-CBSD by Xiangen Hu https://padlet.com/xiangenhu/8iyypn14slxdtxxc What are the xxxxx to Intelligent Tutoring Systems in your assigned research area? en-us 2021-08-31 21:17:49 UTC 2023-05-02 18:09:02 UTC hello@padlet.com ITS can simulate authentic environments. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728713729 2021-09-09 17:01:15 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728713729 ITS are designed to report data. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728715106 2021-09-09 17:01:47 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728715106 ITS can alter scenarios (statically/dynamically). -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728717225 2021-09-09 17:02:36 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728717225 Time and expense to create scenarios -including GIFT. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728719376 2021-09-09 17:03:28 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728719376 ITS assess outcomes (data translation is needed). -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728722817 2021-09-09 17:04:21 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728722817 Authors want to report on their own frameworks. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728725587 2021-09-09 17:05:01 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728725587 Tools for linking ITS to competencies. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728727033 2021-09-09 17:05:32 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728727033 Creation of predictive models using ITS data. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728729090 2021-09-09 17:06:17 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728729090 Tools that enable the framework to scenario pathway. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728731647 2021-09-09 17:06:53 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728731647 IoT and other systems can report data (more easily). -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728733235 2021-09-09 17:07:28 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728733235 ITS are viewed as tools for education, not training. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728734837 2021-09-09 17:08:06 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728734837 Bad predictions could sour the market. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728736144 2021-09-09 17:08:36 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728736144 Build a robust competency data pipeline and feedback loop. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728739409 2021-09-09 17:09:49 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728739409 Demonstrate value. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728740374 2021-09-09 17:10:09 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728740374 Build tools. -Robby Robson xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728741137 2021-09-09 17:10:27 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1728741137 Higher order/composite constructs compared to KSAs are aligned with job requirements-they are what managers care about. - Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749099724 2021-09-17 16:07:40 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749099724 Performance assessments, which naturally pair with competencies (vs. MC, ratings), tend to provide strong validity evidence (e.g., content, criterion). -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749104546 2021-09-17 16:09:50 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749104546 Performance standards, (performance levels, performance level descriptors) enable clear communication with stakeholders. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749118009 2021-09-17 16:15:41 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749118009 Competency scenarios can be idiosyncratic (specific), therefore danger of training irrelevant features rather than the broad competency-must develop many diverse scenarios to avoid specificity (increasing development cost). -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749285294 2021-09-17 17:30:02 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749285294 Performance assessments are expensive (development, administration time) and may be unreliable due to specificities, or alternatively take a lot of user time to assess. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749289587 2021-09-17 17:32:08 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749289587 ITS approaches have not yet proven advantageous. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749291277 2021-09-17 17:32:57 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749291277 Many valued competencies are "soft skills" a relatively untapped training domain. There is a potential first-mover advantage. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749297723 2021-09-17 17:36:12 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749297723 A first-mover advantage is especially true for ITSs--ITSs have traditionally serviced hard skill domains. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749300527 2021-09-17 17:37:37 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749300527 Student, domain modeling have not yet been attempted in soft skill areas--we're early on the growth curve--lots to be learned yet with benefits to learning science beyond ITS itself. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749305897 2021-09-17 17:40:17 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749305897 Market consolidation (e.g. Blackboard-Anthology) will drive out competition and innovation; since ITS haven't proven value yet, traditional approaches will reign. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749309525 2021-09-17 17:41:59 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749309525 Expense of competency-based scenario design will favor adoption of less expensive, traditional approaches. -Patrick Kyllonen xiangenhu https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749314392 2021-09-17 17:44:15 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749314392 https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749772927 2021-09-18 00:11:30 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1749772927 GIFT as a Portal for ITS robbyrobson https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1754698292 GIFT is being used as portal for simulations and other training systems, but imposes its own paradigm (e.g. evaluating performance at, below, or above expectations). This raises the question of whether it could support several different general approaches and act as a unified portal or middleware, connecting to, say, 90% of real world training and education systems. I can see some market advantages if it can do so.]]> 2021-09-20 16:50:03 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1754698292 patrickkyllonen https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755930751 Develop additional measures of soft skills--Situational judgment tests, real-effort tasks, collaborative problem-solving tasks, time-, risk-, and social-preference tasks]]> 2021-09-21 02:59:50 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755930751 patrickkyllonen https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755933083 Implement Question and Test Interoperability Version 3.0 (QTI v3) specification (IMS Global, 2020) for storing and exchanging items and tests, deploying item banks]]> 2021-09-21 03:01:17 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755933083 patrickkyllonen https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755934561 Implement complex performance measures--Mock interviews, Role plays, Character critiques ]]> 2021-09-21 03:02:15 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755934561 patrickkyllonen https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755936208 Implement technology for scoring ill-defined, subjective, and complex tasks 
•Video record trainee’s performance (e.g., interview, role play) following a prompt
•Retrievable for scoring using playback (e.g., pause, fast-forward) and annotation and input tools (e.g., for comments, ratings). 
]]> 2021-09-21 03:03:15 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755936208 patrickkyllonen https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755937583 Long-term, implement real-time automated scoring based on stimulus features ]]> 2021-09-21 03:04:07 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755937583 patrickkyllonen https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755941881 Build a “soft skills” competency training system and get it into the field—determine what works, get user feedback
Widespread data collection will enable big data evaluation approaches—what is the nature of successful performance? What do good performers do? What processing steps do they take? ]]> 2021-09-21 03:06:43 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1755941881 Question jeanine_defalco https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1763870924 Zhao and Watterson (2021) say the following: "For humans to thrive in the age of smart machines, it is essential that they do not compete with machines. Instead, they need to be more human. Being unique and equipped with social-emotional intelligence are distinct human qualities (Zhao 2018b, 2018c) that machines do not have (yet). In an AI world individual creativity, artistry and humanity will be important commodities that distinguish us from each other." Do the soft cognitive skills implicated in these commodities differ from traditionally thought of soft cognitive skills?

]]> 2021-09-23 19:00:08 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1763870924 Build Authoring Interfaces to Support Competency Modeling https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1763912538 I agree with Robby... oh my... I can't believe I said that out loud... but we need to make it easier to develop competency models and translate competencies across different experiences (training, job). An interesting trend is open credential publishing.  These systems are supported as individual wallets that can be shared in different training and educational environments by the learner.  Standardizing interfaces for enable competency model data to be developed, shared, and consumed by other ITSs will go a long way to reducing workload associated with building competency models.

]]> 2021-09-23 19:22:31 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1763912538 The Task of Documenting Required Skills and Knowledge https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1763921303 As a basis for generating learning objectives or concepts to learn, the task of documenting skills and knowledge required to do a job on a day to day basis is difficult... massively difficult.  It often requires many SMEs to determine the requirements and then the set of experiences needed to develop need knowledge and master essential skills.  The more we can distribute the workload of this process the more prevalent and accurate competency modeling will be.

A working group to help solve this problem is needed.]]> 2021-09-23 19:27:21 UTC https://padlet.com/xiangenhu/8iyypn14slxdtxxc/wish/1763921303