# data[].attributes.body
Bulk upload UoAs to Person records to improve reporting of reviews against units of assessment

# data[].attributes.body
Update reporting module to allow individual questions in a review template to be reported on rather than the question type

# data[].attributes.body
Add configuration on visibility of reviews, to be visible for all authors or not

# data[].attributes.body
Add setting to show/hide the propose reviews section on the personal overview page.

# data[].attributes.body
Ability to change the managing organisation of the reviews, so they can be easily updated and or aligned with UoAs

# data[].attributes.body
Bulk Upload: Crucial to be able to bulk upload reviewers to the requested reviews

# data[].attributes.body
Bulk Upload: request reviews via a bulk upload, with the option to include a reviewer and review deadline (Delivered)

# data[].attributes.body
Saving review requests: Users unable to see or remember to use the save Button (mitigated to some extent with the new personal overview screen)

# data[].attributes.body
Visibility of data: final grade and comment in a Review is visible to the requester prior to the review being 'finalised'. The Review summary screen doesn't appear to respect the finalisation status of the review (Fixed)

# data[].attributes.body
Managing reviews: editors find it difficult to see where all the reviews are, what steps they are at, which reviewers have completed or not completed a review (Resolved with the new personal overview screen)

# data[].attributes.body
Grade uploader doesn’t mirror the review template settings, resulted in a manual edit and activation of the grade field for every reviewed output. (fixed)

# data[].attributes.body
Users frustrated by not being able to edit reviews after saving or submitting (Fixed via recall review function)

# data[].attributes.body
Information provided by the review requester does not automatically get added to the review template and results in a lot of copying and pasting for administrators/editors.

# data[].attributes.body
Limit reviews to publications after the start of the current REF period (1 January 2021); this is resolved in the personal overview function, but it doesn’t stop people proposing from the publication template for outputs published at any time

# data[].attributes.body
Change proposing author - It is now possible to change the proposing authors on reviews. This can be especially useful if the proposing author have left the institution, or it was done by an administrator. The proposing author can be changed by clicking on the pencil next to the proposing author on the review editor.

# data[].attributes.body
Email notification settings not always working for individual reviewers, links in emails not always working as expected

# data[].attributes.body
No guidance as to how we should be using the different final statuses (approved, not approved, approved with conditions, done. Make final Review statuses accessible and editable so that institutions can use these flexibly

# data[].attributes.body
status of reviews not clear for reviewers. For the reviewer it can be difficult to see what is pending and what’s been done (without opening the record); especially when there are multiple reviewers, you can’t tell as an individual reviewer if another review is outstanding without opening the review (to some extent delivered with the review summary screen)

# data[].attributes.body
Proposing outputs, the free text box doesn’t allow institutions to capture enough structured data

# data[].attributes.body
REF open access compliance status

We have added information about the REF open access compliance status in the Review editor. This was a requested update to help process the review requests.

# data[].attributes.body
Make the author and co-author details visible in the related content list on the metadata tab of the review setup to avoid allocating a review to a co-author

# data[].attributes.body
Allow the upload of documents or URLs to the review, that are not on the output template, e.g., drafts of book chapters that have not yet been submitted, or a link to SharePoint folder. It would be useful to streamline the process by allowing 100% of the review process to be done on Pure.

# data[].attributes.body
Editors are often also reviewers, but when an editor they cannot carry out a review. Review editors need a sensible way to be able to carry out reviews.

# data[].attributes.body
Where UoA has been added to a person, this should be visible on the review screen, as it will be really useful when amending managing org of reviews.

# data[].attributes.body
Make the description text in the ‘Accept review’ box editable

# data[].attributes.body
Make saved reviews visible in the same screen as other reviews. Create summary screens for reviewers that clearly indicate reviews for completion and reviews that have been saved for later (Delivered with the personal overview summary screen)

# data[].attributes.body
Allow editors to add the same person to a review again if an earlier review has been completed or rejected. This would allow a clearer audit record of how the review was carried out.

# data[].attributes.body
Grade uploader has been temperamental, and more guidance is needed; Improve guidance on how to use the grade uploader

# data[].attributes.body
Identify clear roles and permissions for unit or Organisation levels. Improve guidance on the roles associated with reviews. Global access to report on reviews should not be required if users only need to report at unit level.

# data[].attributes.body
Design/User experience, Bulk Editing: So many clicks to assign reviewers and get the reviews out to people. The bulk upload will help (hopefully), but it’s an extra admin step for us. Bulk operators in the systems would be useful. There tends to be only a small pool of reviewers so bulk assigning 10+ outputs to a reviewer would be useful.

# data[].attributes.body
Design/User experience: Review message after submission cannot be changed. Admin/editor access to edit this message would be useful.

# data[].attributes.body
Reporting: An export to Excel on the Review list screen would be nice for easier workarounds at UoA level.

# data[].attributes.body
Reporting: Reporting of Reviews permissions would be useful, especially while there are no summary screens. Need to be admin or global reporter to receive snapshots. This is why institutions have had to create PowerBI dashboards.

# data[].attributes.body
To apply final grades and comments, the entire bulk uploader has to be used – essentially overwriting content in the system already. Potential risk for error. Ideally bulk upload could be used just to add in final grades and comments.

# data[].attributes.body
No way to bulk disable or enable notifications to reviewers about reviews.

# data[].attributes.body
When the reviewers award different ratings, it would be useful to have some functionality to notify both reviewers to ask them to try to agree a final rating from the Responses tab (the notification should include both reviewers rating & comments so they can discuss and agree a final grade).

# data[].attributes.body
It would be useful to be able to add another template to the same output... e.g. if re-opening a review because review question requirements have changed, or if multiple templates are in use and the wrong template has been assigned

# data[].attributes.body
There is an error where if you assign the reviewers before the template, it locks the reviewers’ field.

# data[].attributes.body
Terminology varies in reporting and the review module - The creator of the review is the proposer and then created by is shown in the table but in the filters its requested by if you are filtering on the module.

# data[].attributes.body
There is a bug that prevents adding certain documents from the output to the review depending on the validation status of the output

# data[].attributes.body
If you update the guidance text on the REF29 template it doesn’t update when the template is applied to a review – the template has to be manually assigned for any changes to take effect (this is resource intensive with some institutions having to do this manually for thousands of records)

# data[].attributes.body
Only outputs in ‘Published’ status appear in the ‘Propose outputs for REF 2029’ functionality on Personal profile.

# data[].attributes.body
No REF1/REF2 structure to work towards

# data[].attributes.body
No connection between reviews and REF 2

# data[].attributes.body
No summary screens so institutions have had to build PowerBI dashboards. Essentially institutions have made REF1/REF2 summary screens for themselves, but it requires lots of effort to keep it up to date. Example dashboards from institutions:

a)       Show people and their UoAs

b)      Show who has proposed reviews and who hasn’t

c)       Track progress by reviewers

d)      Show major differences in grades

e)      Looks at co-authors and shows if their outputs have been proposed by other co-authors

f)        Show a summary of outputs being reviewed across the UoAs

g)       Importantly – shows all the org mismatches where a UoA lead of not able to view and assign reviewers to outputs proposed in their UoA.

h) Results of reviews – showing grades, grade discrepancies between reviewers, summaries per UoA, former staff outputs, status of reviews – would extend to OA tracking in the future.

# data[].attributes.body
No Reporter of Output review function at organisation level. Currently global reporter must be applied and that is problematic.

# data[].attributes.body
Impact templates - Suggestions from the group on additional custom templates are a big chunk of work and unlikely to progress for REF 2029.  Institutions who have not already customised the impact module templates are encouraged to think about updating the Types of Impact classification scheme “/dk/atira/pure/impact/impacttypes” to create additional named templates to improve filtering and reporting options on Impacts which are focused on REF impact case studies

# data[].attributes.body
Review on Impact is available from 5.32.0 but needs to be turned on by request

# data[].attributes.body
CKU Workflows: bulk and manual creation of REF 2 records for institutions who are and are not using reviews

# data[].attributes.body
Units of Assessment on Review records

# data[].attributes.body
Multiple REF 2 Records based on UoA

# data[].attributes.body
Skeleton operable REF 2 workspace that connected to output reviews where these have been used or uploaded. Key components for a skeleton record included workflow for pool selection, Unit of Assessment, Cross referral indicator, double weighting indicator, overall grade, review comments and grades, open access status, internal author list with notes box

# data[].attributes.body
Ability to upload of PDF documents to REF 2 Records where they may not be suitable to add to the output record.

# data[].attributes.body
REF 2 elements specific to REF 2029

Author information and person association including contract details and units of Assessment

Significant contribution / substantive connections

Tracking of person and PCE elements for EDI analysis and narrative statements

# data[].attributes.body
REF summary screens and reporting

track PCE and person elements connected to REF 2 records

report on all available fields

Clear summary screens similar to those available in REF 2021

filter REF 2 records by Unit of assessment and other criteria

report on outputs, how many REF 2 records are linked to them and which Units of Assessment they are in.

# data[].attributes.body
Possible use of an algorithm to optimise output selection

# data[].attributes.body
Possible use of calculations or notifications to identify outputs with & without substantive connection

# data[].attributes.body
Ability to manage merge or delete REF 2 records if too many are created

# data[].attributes.body
Staff data in REF 1 necessary for understanding the REF volume measure and FTE calculations, which underpinn output and Impact case study requirements.

# data[].attributes.body
REF 1 records help institutions clearly record staff on eligible contracts and verify the substantive connection of eligible outputs by tracking the interaction between staff and outputs

# data[].attributes.body
REF 1 record should have

one REF 1 record for each added person

REF 1 record that allows the allocation of multiple Units of assessment

FTE values based on contract level data

contract level and total FTE override options

# data[].attributes.body
REF 1 records need to

link to the person record

allow institutions to identify if a person is in scope for REF

allow institutions to identify if a person is contributing to the volume measure

allow allocation of multiple Units of Assessment

capture FTE values at contract level by UoA, with optional override values

feed volume contributing FTE values to Units of Assessment

be reportable

link to REF 2 records to track interaction between staff and outputs and the substantive connection to an output.

# data[].attributes.body
Possible use of standardised keywords to record the HESA data

Included in REF volume measure

REF Eligible or Qualifying contract

Early Career Researcher

Research Independence

# data[].attributes.body
Ability to bulk upload or bulk create REF 1 records in Pure with data on REF eligibility, inclusion in volume measure, and FTE

# data[].attributes.body
API connections for REF data

# data[].attributes.body
bulk upload information into a REF 1 module

# data[].attributes.body
Description boxes - Institutions who have not already customised the impact template are encouraged to update the classification “/dk/atira/pure/impact/descriptions” to increase the number of description boxes available in the Impact template

# data[].attributes.body
Status & Workflows - general impact statuses, not specifically focused on REF, would make sense to develop against the existing impact workspace

some workflows and Statuses, related to REF, may be better suited to the REF 3 workspace where specific REF fields and workflows could be developed

# data[].attributes.body
Evidence files - changes to evidence upload would be a change to the fundamental process of file handling in Pure. Elsevier noted that they would need to talk to colleagues responsible for file uploads to see what was possible or feasible

# data[].attributes.body
REF 3 workspace and Impact template to have a much stronger connection and improved linking between these areas

# data[].attributes.body
Summary Screens / Reporting

overall list of case studies.

Summary screens/Overview screens to help track progress

Better reporting functionality, including impact review and scores

Summary screens (and reporting) helpful in pipeline tracking and gap analysis, including identifying where evidence files were missing

# data[].attributes.body
diary style function could be possible

# data[].attributes.body
milestones and deadlines in impact were reasonable considerations

# data[].attributes.body
Some notes and administration functions could be managed in Impact reviews

Impact reviews utilising deadlines could be utilised as part of impact development

# data[].attributes.body
documents can be added to impact reviews but evidence files cannot