FAQ

Frequently Asked Questions

The questions and answers presented in this section reflect actual enquiries and situations from previous NCMAS calls. The Committee and Secretariat recommend that all applicants read this FAQ section in full.

General

1.0    I have a question about NCMAS 2022 and can't find an answer in this FAQ. How do I get more information?
Email ncmas@nci.org.au if you have any questions about NCMAS. This email will reach the Secretariat, who will contact you with further information.

1.1    What is the deadline for application? 
The application deadline is 8:00pm AEDT (5:00pm AWST) Tuesday 5 October 2021.

1.2    Can I request an exception to the application deadline if it overlaps with a work commitment, such as a conference?
Unfortunately, no. If you have commitments at or near the deadline you are advised to submit your application as soon as possible during the call.

1.3    If I have a draft application in the NCMAS system at the time of the submission deadline will I be allowed extra time to finish and submit my application?
No. Applications cannot be submitted after the deadline.

1.4    Can I resubmit my application from the previous NCMAS call (2021) with minor changes and updates?
Yes. 
However, resubmission of an application as-is from the previous call without updates to reflect progress is not recommended and should be reviewed for compliance. Your application from the NCMAS 2021 round can be viewed at  https://my.nci.org.au/mancini/ncmas/2021/.
Note, also, that all CIs on your project should update their career profiles and research track records to reflect recent developments or changes.
Compliance for NCMAS 2022 includes anonymous third-person style for the Proposal and Computational Details components.
Applicants can submit these components for an optional pre-submission review by the NCMAS Secretariat through the NCMAS form to receive feedback before final submission – due 7:00pm AEST (5:00pm AWST) 13 September 2021.

Grants

1.5    The outcome of my ARC or NHMRC grant proposal is expected to be announced during the NCMAS call. How do I include this outcome in my NCMAS application?
Research funding outcomes announced in the September-December time frame are relevant to your application. 

  • You may submit an addendum to add recently announced funding via email to ncmas@nci.org.au until the Merit Assessment begins.

1.6    What period is covered by the “previous five (5) years” requirement for publications and grants?
The “previous five years” period extends from 1 January 2016 to the present, inclusive.

Lead Chief Investigators, Chief Investigators, Researcher roles

1.7    Is it possible to transfer my project to another Lead Chief Investigator?
Yes. The recommended process is:

  • The new Lead CI should register and join the project using the https://my.nci.org.au system.
  • If you (as current Lead CI) wish to leave the project, you should promote another project member to the Delegate Lead CI role, and then leave the project. Leaving the project will elevate the Delegate Lead CI to the (full) Lead CI role. 
  • If you (current Lead CI) wish to remain in the project in a (regular) CI or Researcher role then submit an NCI support request to change your project role. The support request can be submitted by email to help@nci.org.au. Please provide the project code, your request and the new Lead CI’s information in the support request.

1.8    I’d like to include as many researchers as possible on my project to demonstrate its significance and impact. Can I include former members of my group, with whom I still collaborate?
A Lead Chief Investigator should update their project memberships to include personnel who will actively contribute to the project during the next calendar year.

  • Inclusion of former group members who will no longer contribute to the research is not recommended. 
  • Retention of a previous Chief Investigator could inadvertently breach eligibility rules if that CI is now listed on another NCMAS application.
  • The key rule to observe is: An investigator can be named as a Lead CI or (regular) CI on one project only

1.9    I am a research student, and my supervisor has asked me to complete the NCMAS application for our research group. What do I do?
Your supervisor needs to change your project role to Delegate Lead CI to enable you to access the online NCMAS application form. 

  • Your supervisor can do this through the NCI registration system (https://my.nci.org.au) or by emailing NCI support at help@nci.org.au.
  • Note that as Delegate Lead CI you become the responsible agent for the application. You will need to keep your supervisor informed through the call for applications period.

1.10    Why do I need to update memberships of my NCMAS project?
The members of you project should include only those who actively contribute to the project. Prior members of you research team who will no longer contribute should be removed from your project. 

  • Note that removing former team members is also required under Conditions of Use for NCI and other HPC facilities.

1.11    My research team includes people who will not be involved in computational work for my project. Should these non-computational team members be included in my NCMAS application?
Team members who are not part of computational activities are not required to be listed on your application. 

  • Note that if you do wish to include such persons, they will need to register for an NCI user account and join your project.

Application process, proposal components

1.12    Does the word length limit for the Proposal apply to both the Proposal body and references?

The Proposal should be no more than five (5) pages long, excluding references. 

  • Proposal and Computational Details should be combined into a single PDF file to be uploaded.
  • The online form does not enforce a word limit in the PDF upload. 

1.13    Can I request less than 500 kSU per year in the 2022 NCMAS call?
Yes – if you are requesting resources on Setonix or MASSIVE. 

Facility-System Minimum Allocation (kSU/year)
NCI-Gadi  500
Pawsey-Setonix 1st Request  250
Pawsey-Setonix 2nd Request  1000 (pro rata)
MASSIVE   50

 

Applicants with smaller requests are advised to contact facility and/or partner allocation schemes. 

1.14    Can I attach my CV instead of entering publication and grant information on the application form?
No. It is not possible to attach a CV to the application. Please enter relevant publication and grant information on the online form.

1.15    Our NCMAS Proposal contains graphs and images. How do I include figures in my application?
Submission of the Proposal and Computational Details via PDF upload in your application gives you more control over proposal format and content, including figures and graphs. 

1.16    My project requires HPC resources for two types of work:
           1.    Research and development, and 
           2.    Ongoing operations, for example, a data analysis service. 
           Can I use NCMAS to obtain resources for both types of work?
NCMAS should be used to obtain resources for your research and development work. 
Facility and/or partner schemes are usually the appropriate resourcing channels for operational or service HPC workloads.
 
1.17    Can I modify or correct errors in my NCMAS application after submission?
Submission is considered final, as noted in the Information for Applicants, so an application cannot be modified following submission. 

  • You may submit an addendum to add recently announced funding via email to ncmas@nci.org.au.

1.18 How can I make my application more competitive?
First, make sure you are applying in the category appropriate to your level of experience. 

  • Postdoctoral fellows who are establishing themselves in the field are advised to submit an ECR application. 

All applicants should read the Information for Applicants in full, pay attention to detail, and demonstrate to the fullest extent their ability to use national facility HPC resources at scale.

1.19    Why is the application process so complex?
NCMAS is very competitive and resources are limited. Applications must capture detailed information needed by the Committee to assess scientific merit and the ability to use HPC resources at scale. 

1.20    Why don’t I receive detailed feedback about my application?
NCMAS receives approximately 250-300 applications in each yearly call. Committee workloads are such that detailed feedback for each application is not possible; each member of the NCMAC will assess between 20 and 50 applications. The NCMAC will, however, try to provide brief suggestions to the applicant in as many cases as possible. The NCMAC will also record comments raised during the allocation meeting for inclusion in your outcome.

1.21    Can I get advice on how to write my NCMAS application?
Unfortunately, the Secretariat and Committee are not able to provide specific advice to individual applicants. All applicants are advised to:

  1. Make sure to read all supporting documentation, including rules and guidelines.
  2. Read through the example Proposal and Computational Details provided – this was a highly ranked submission from the 2020 round.
  3. Check your eligibility. If you have questions about eligibility contact the Secretariat as soon as possible.
  4. Update your profile and research track record and contact the CIs in your research team to ensure they have completed updates.
  5. Make a compelling case for your proposed research.
  6. Demonstrate your ability to use the national HPC facilities at scale.
  7. Include references for all research funding over the previous 5 years.
  8. Submit your application through the optional compliance check for A3P. It is the responsibility of the applicant to make any suggested changes before final submission
  9. Write clearly. Pay attention to detail.
  10. Respect the guidelines and the submission deadline.
  11. Review the example application provided

1.22    My NCMAS 2021 application was unsuccessful. Why didn’t I receive an allocation?
In most cases NCMAS applications are not successful for one of the following reasons:

  1. Eligibility issues;
  2. Non-compliant or incomplete application;
  3. Poor conception or development of proposal;
  4. Insufficient demonstration of peak-scale HPC requirement;
  5. Inadequate justification of HPC resource request;
  6. Ambit claims for large-scale resources.

New applicants to NCMAS should, if possible, demonstrate a track record of effective HPC utilisation through partner or facility resourcing schemes.

1.23    Why do some projects receive large NCMAS allocations?
NCMAC will consider allocation of more than 5 MSU/year to applications which demonstrate exceptional and sustained track records or HPC utilisation, and which make a compelling case for HPC resources at large scale. 

  • Projects in this category typically present well documented resource requests with compelling justification, and with extensive details of methodology, workflows, application performance, and scalability. 
  • These projects also have strong records of successful grant proposals and research output.

1.24    What is the proper length and level of detail for a NCMAS Proposal?
    Proposal and Computational Details length limits are described Information for Applicants document. 
Proposal Length
Proposal should be no longer than five (5) pages (excluding references). Providing historical context and ten-year scope of the research should take no more than one (1) page.
Computational Details Length
Computational Details should be no longer than ten (10) pages (excluding references).
PDF requirements
Proposal and Computational Details must be provided as a PDF upload (one file only – both components should be included). 

New projects

1.25    Is the project description in the NCI new project application the same as the Proposal for NCMAS in the application form at https://ncmas.nci.org.au?
No. 
The description in the NCI new project registration (at https://my.nci.org.au) is treated as a project abstract for the purposes of NCMAS. 
The full, detailed proposal must be submitted in the NCMAS online application form – https://ncmas.nci.org.au.

1.26    Does the NCI new project request submitted through https://my.nci.org.au need to be approved before the project is available in the NCMAS application system at https://ncmas.nci.org.au?
No. 
New projects registered for NCMAS at https://my.nci.org.au remain in “pending approval” state until their NCMAS outcome is determined.
NCI will approve project registrations for successful applications after NCMAS outcomes are finalised. 

1.27    Last year I made a mistake and completed a project registration in the https://my.nci.org.au system but did not complete the full NCMAS proposal in the https://ncmas.nci.org.au system and missed out. 
How can I make sure to complete a full application in the 2022 call?
Read the instructions in the Information for Applicants document in full and ensure that you complete your new project registration and your full merit proposal. 
Contact ncmas@nci.org.au if you run into any difficulties during the application process.

1.28    How do I use the ORCID functionality in my NCMAS application?
Applicants must import publication references from ORCID via the interface in the NCI online registration system, https://my.nci.org.au. The ORCID interface allows the user to nominate publications from their ORCID record for inclusion in an NCMAS application. 
(Note that the https://my.nci.org.au ORCID implementation gathers data from ORCID only; it is not an alternative tool for managing your ORCID record.)
Detailed instructions for using ORCID are provided in supporting documentation via the NCMAS website. 
ORCID will be the only method for managing publications in your NCMAS application.
All applicants should read supporting documentation for ORCID functionality in full.   

NCMAS Form 

1.29    Can I add/update project members to my NCMAS application after submitting?
No.
The NCMAS system will take a snapshot of the project membership at the time of application submission. Adding/updating project members after the submission will not be reflected in the NCMAS application. 

1.30    Can I add/update career profiles and publications for all CI's to the NCMAS application after submitting?
No.
The NCMAS system will take a snapshot of the project members' career profiles and publications at the time of submission. Adding/updating career profiles and publications after submission will not be reflected in the application. 

1.31    How can I request an A3P compliance check for my draft Proposal and Computation Details document?
A 'Request a compliance check' button is located on the 'My Application' tab in the online submission form. The NCMAS Secretariat will provide feedback on whether your writing style complies with A3P requirements.
Compliance checks must be submitted by 7:00pm AEST (5:00pm AWST) 13 September 2021. 
It is the applicant's responsibility to adopt any suggested changes before final submission. 

1.32     I have drafted a new project proposal for the NCMAS scheme (proposal-XXXXXX) and want to add people to this project proposal. How do I add people to my new project?
Anyone who wants to join the new project proposal (proposal-XXXXXX) should visit https://my.nci.org.au/mancini/project/proposal-XXXXXX/join to request the project membership. Then the Lead CI of this proposed project will be able to approve the membership request using the https://my.nci.org.au system.
However, please note that, this needs to be done before submitting the application as adding/updating project members after the application submission will not be reflected in the submitted NCMAS application.

1.33    I am a Delegate Lead CI and have discovered the project I'm applying for no longer has a Lead CI. What should I do?
You should get a suitable Lead CI on the project as soon as you can. You will not be able to submit an application for the project unless the Lead CI role is filled.
As a Delegate Lead CI you can use the invitation function in the https://my.nci.org.au system to invite a Lead CI on board, or ask the new Lead CI to register at https://my.nci.org.au and request to join the project. You will have privileges to approve the new Lead CI's membership.

1.34    In my MyNCI Career section there is a place to provide my "Research FTE % - Last 5 Years". Should I include or exclude career interruptions?
This section is to provide the Committee with a quick understanding of your research 'potential' across the last 5 years. You should include career interruptions if they impacted your research. 

E.g. If you did not conduct research at all for 2020 and 2021, and were only employed to do 50% research in the other 3 years, your total would be 30%.

1.35    I want to provide details of graduate student income in the Funding section of the application form. What should I consider?

The intention of this section is to allow a place for describing the support of the research if that support comes in the form of graduate/doctoral students. There has been feedback that many research projects are supported by the students who work on a project, and previously there was no way to reflect this contribution in the NCMAS application system.

  • We suggest you provide details accurately and transparently in your estimation of the dollar value.
    • To calculate the dollar value, you could multiply their annual stipend rate by the number of years they will work on your project
      • If you are 'sharing' the student with another project, please consider dividing the value of the scholarship accordingly
    • If the scholarship is part of your team's work, you can list yourself as the Investigator, and provide the name of the student in the funding title.
    • If the student has their own scholarship, e.g. through a philanthropic grant, you could provide an estimate of the in-kind contribution they are making to your project
  • If the student's work is funded through grants you have otherwise listed, you should not list the funding again
  • If you wish to provide additional context, you can do so in the 'Additional Funding' text box.
    • As this is the first year with this funding type being available, we acknowledge the system may not perfectly capture all details.

1.36    In previous NCMAS rounds there was a a section of the form for a Progress Report. I can't see that section of the form this year. Where do I report on my research progress (publications/datasets/awards) as a result of a 2021 NCMAS grant?

You can use the 'Career' and 'Publication Record' sections of the MyNCI (https://my.nci.org.au) 'About me' profile to provide details of significant publications from 2021.

If datasets produced weren't part of a publication, you could consider describing them in the 'Research Performance Evidence' section of the Career tab, or in the box that asks you to 'Describe any other notable research outcomes of significance or value'.

If you received awards or similar, you can use the 'Non-publication Research Performance Evidence' section of the 'Career' tab.

The Progress Report has been detached from the NCMAS application. In the 2021 round documentation, it was stated that if you were successful in your NCMAS 2021 application, you will be required to submit a Progress Report in Quarter 1 of 2022.

The same timeframe will exist for successful applicants in the 2022 round - a Progress Report will be required in Q1 of 2023 - see page 24 of the current Information for Applicants document. 

1.37     Am I allowed to cite preprints in the Proposal?

You are allowed to cite publicly available preprints. Note that you must still reference in an anonymous third-person style in the Proposal and Computational Details. See Page 5 of the Anonymisation Guide, which discusses the potential issues of implied knowledge of a paper's submission status:

  • Terms like ‘submitted’ or ‘preliminary’ imply knowledge of the publication status of a paper, which could in turn imply you are an author
  • “….a further paper ‘Molecular movement in monotremes’ has just been submitted to Nature…" 
    • This could be identifiable and self-referencing because of the implied knowledge of the submission/publication status of the paper
  • The exception to this if the paper is available and searchable online in pre-publication state.

1.38     The NCMAS form states that "The length and rigour of the proposal should reflect the level of resourcing requested" - what does this mean?

In previous NCMAS rounds the length of the Proposal and Computational Details was proportional to the request size.

The 2022 round has strict upper page limits:

  • 5 pages for the Proposal (excluding references)
  • 10 pages for the Computational Details (excluding references).

However, there is still capacity for smaller requests to write shorter documents.

 

Allocations

2.1    I was unable to utilise most of my 2021 NCMAS allocation due to unforeseen circumstances or external dependencies. Can I still apply to NCMAS for 2022?
Yes. Your application should clearly explain the circumstances leading to under-utilisation of your 2021 allocation. Your application should provide a strong justification of your 2022 resource request.
If your project has used less than 90% of your 2021 allocation at 2021 Q3 (pro rata), you should provide an explanation for underutilisation of your current allocation.

  • Unjustified underutilisation may impact negatively on the merit of the application.

2.2    My project expects to use its NCMAS allocation at specific periods during 2022, and to have some corresponding periods of low usage. Can I request that my allocation be provided non-uniformly across quarters in 2022 to accommodate this operational requirement?
Non-uniform installation of a compute allocation is at the discretion of the HPC facility. Generally, facilities can accommodate small variations in quarterly allocations. They will not, however, be able to install an allocation into just one or two quarters, for example. 
Applicants who have scheduling dependencies and expect to have seasonal or varying usage are advised to engage directly with the facilities to discuss their options.

2.3    In 2021 I consumed my allocations before the end of each quarter. Can I request supplemental allocations if I face a similar situation in 2022?
No. Unfortunately, NCMAS is unable to provide supplemental allocations because it is heavily oversubscribed. Supplemental allocations are best sought from partner schemes if additional resources are needed during the year.  

2.4    Why have my previous NCMAS allocations been less than the amount requested?
NCMAS allocations are highly competitive; demand for SU has exceeded supply by a factor of 2-3. The Committee determines each allocation based on the merit of the proposal and track record of the project and its CIs. The Committee must also adjust allocations to attempt to optimise usage of the NCMAS resource shares.

2.5    Can I appeal if I am not satisfied with my NCMAS outcome?
Allocation decisions by the Committee are final. Appeals are accepted only in cases of administrative error on the part of the Secretariat, Facilities or the Committee. Administrative appeals are decided by the NCMAC Chair, with the assistance of the Secretariat. 

2.6    Can I apply for a Pawsey/Setonix allocation and use other Pawsey resources?
Resources are allocated on the nominated NCMAS systems via the NCMAS allocation application process.
The facilities do have access to other technologies and services, and access to these resources may be requested if needed. These other systems may be leveraged after the allocation on the main systems has been awarded if there is capacity and the researcher requests to do so with the facility.

Eligibility

3.1    I am a research student, and my supervisor has asked me to submit an application to NCMAS. What should I do?
As a research student you are not eligible to apply for NCMAS. You should advise your supervisor to check their eligibility and apply.

3.2    As a Lead CI, I want to delegate preparation of my NCMAS application to a senior member of my research group. How do I do this?
Promoting a team member to a Delegate Lead CI role on your project will grant them access to the NCMAS online application system. Promotion is done through the online registration system at https://my.nci.org.au:

  1. Log in to https://my.nci.org.au
  2. Go to your project listing (click on project name - in blue)
  3. Select your delegate's name (click on name)
  4. Use the Change tab to change the person's role to Delegate Lead CI
  5. Submit the change.

3.3    I am a Lead Chief Investigator on my own project, and I also collaborate closely with a research group at another university. Can I be included as a (regular) Chief Investigator on my collaborator's NCMAS application?
No. 

  • An individual can be a Chief Investigator or Lead Chief Investigator on one NCMAS application only. 
  • Membership as a Chief Investigator or Lead Chief Investigator in more than one NCMAS application will render all such applications noncompliant.

3.4    I have recently started a Postdoctoral appointment and wish to submit an application to NCMAS. What should I do?
You are eligible to apply to NCMAS, however, you will be expected to demonstrate that you have independent research funding support, such as an ARC DECRA or similar award. 

  • Applications which do not provide evidence of independent research funding will be disqualified before merit assessment. 
  • There is also an expectation that Lead CIs with recent Postdoctoral appointments will apply in the Early Career Researcher category.

3.5    I have recently resumed my research career after a period of interruption. Am I eligible to apply to NCMAS?
You are eligible to apply in the Special Consideration category provided your research work has resumed within the last five (5) years, and your PhD was awarded within the previous nine (9) years. 

  • Please also note that Special Consideration applications are expected to demonstrate independent funding support.

3.6    I am a Research Scientist employed at an Australian Government science agency (e.g. ANSTO, BoM, CSIRO, DST, Geoscience Australia). Am I eligible to apply?
Yes, however applicants from Australian Government agencies are expected to hold a position of CSOF5 (or equivalent) or higher. 

3.7    Our research group pursues several related compute-intensive research projects, with each research thread is led by a different member of the group. Can we submit individual NCMAS applications for each body of work in the combined research effort?
No. The group should submit a single, combined application to NCMAS. Note that an individual may be Lead CI or CI on one application only, so individual applications submitted in this case would be ruled non-compliant.

3.8    I am an experimentalist who wishes to use NCMAS to supplement my research with theory/computation. Am I eligible to apply?
Yes. Your application should aim to demonstrate HPC expertise of your group and clearly describe how computation will contribute to your research plan. 

3.9    I am a new research faculty/Postdoctoral appointment, but do not yet have a track record of funding support. Am I eligible to apply?

NCMAS applicants are expected to demonstrate a record of independent funding support. NCMAS recommends that you investigate HPC resourcing through your local institution, which may have an active partnership with one or more of the NCMAS facilities: NCI, Pawsey, and MASSIVE. 
A resource allocation from your institution can be a vehicle for development of HPC expertise, with a view toward a future NCMAS application.

3.10    If a Researcher on a project is promoted to a Delegate Lead CI role to manage the project’s NCMAS application, can that Researcher also be a Delegate Lead CI or (regular) CI on another NCMAS application?

No.

  • A person who is a Researcher on two projects cannot be promoted to a Delegate Lead CI role on both projects for the purposes of completing the applications.
  • If any person is a (regular) CI on one project, and a Researcher on another, the Lead Cis on both projects should exercise care about personnel roles.

3.11    A Postdoctoral Fellow will be joining my project in January 2022 (formal job offer has been accepted) and I would like to include them as a CI on my NCMAS application. The future Postdoc will not receive their PhD until December 2021. Can I include this Postdoc as a (regular) CI on my NCMAS application?
A PhD candidate may be included as a (regular) CI on an NCMAS application.

  • Note, however, that the Committee will review research track records of the Lead CI and all (regular) CIs as part of the assessment process. 
  • If the PhD candidate has outstanding research outputs this could be a positive factor for the application. 
  • If there are few or no demonstrable research outputs it might be better for the PhD student to take a Researcher role 

3.12    I have an Emeritus position at my institution. Am I eligible to apply?
A project Lead CI with Emeritus status is eligible to apply to NCMAS.
However, there is a general expectation that Emeritus-led projects will have a relatively modest size team and scope, compared to larger projects with top-tier resource requests.

3.13    Does NCMAS use the term 'ECR' in the same way as the ARC, NHMRC, my university, or other grant agencies?
For NCMAS, the term ECR is used to define a specific application category and set of eligibility criteria. NCMAS does not use the term ECR in the same way as other granting agencies or institutions, so NCMAS applicants should not assume the term has the same meaning. 
All applicants are advised to read NCMAS supporting documentation in full for further information. 

3.14    I am an early career researcher who would like to apply in the ECR category and also be included in my group leader’s NCMAS open category application. Is this possible?
Unfortunately, NCMAS rules state that you can be a CI on only one application.
If you wish to apply for your own ECR project you cannot be listed on your group leader’s project as a CI.
You can, however, remain on your group leader’s project in a Researcher role. As a Researcher, your track record information will not be considered in the assessment process.

3.15    If the exact dates of my PhD and the NCMAS due date are used, I would be considered less than a year (e.g. 9 months) outside the 5-year time frame. How is the previous five (5) years time since PhD is calculated for the ECR category application eligibility?

A PhD awarded in the period from 1 January 2016 to 5 October 2021 would be eligible for the ECR category in the 2022 NCMAS round.  

Anonymisation

4.1    Why is NCMAS using double-anonymous reviews?
NCMAS places a high value on the equity and integrity of the proposal review process. 

The goal is to minimise potential sources of bias. 

Several studies have shown that a reviewer's attitude toward a submission may be affected, either consciously or unconsciously, by the identity of the lead author or principal investigator (for more information - see the Women in STEM Ambassador Research Project).  

Reviews of other resource allocation schemes has shown that proposals led by women have systematically lower success rates than those led by men. 

Studies suggest a double-anonymous process would significantly reduce the potential for bias, whether by gender, affiliation, or country of origin. 

Such a process may also level the playing field between new and established researchers.

4.2    Are truly anonymous submissions even possible?
Even in the relatively small community of computationally- and data-intensive research fields, it is not as likely as one might believe to correctly guess the authorship of a proposal.

While it is possible to correctly guess the authorship, studies from other similarly small fields suggest the Lead CI's identity would remain unknown 60% to 75% of the time. 

So, while a system that provided perfectly unbreakable anonymity would be ideal, our goal is simply to obscure identity, to discourage guessing, and to reduce unconscious bias, and not make authorship a focus of the evaluation of a proposal.

4.3    How will you know if the experiment was successful or not? 
The Facilities do not consider the move to a double-anonymous process to be an 'experiment'. 

It is one in a progression of changes that are being implemented to improve the equity and integrity of the proposal review process. 

NCMAS will continue to evaluate the review process, with attention to fairness and balance over several factors.

If you would like to be sent the published results from the Women in STEM Ambassador's Trial, you can check the box in the consent section of the application form. 

4.4    Is there scientific evidence that anonymising proposals results in a reduction of bias?
Yes. While this process may be new to computationally- and data-intensive research fields, there is an abundance of literature on this topic from many different fields.

The Women in STEM Ambassador's Research Project – National Trial of Anonymising Research Funding Proposals -has reviewed the literature on this topic, and the results are clear: the removal of names and affiliations results in reduced bias in the review process. Read through the FAQs from the National Trial.

Some relevant journal articles on this are available in this Annotated Bibliography compiled for a recent study of NASA’s dual-anonymous review process. 

4.5    How difficult is anonymisation for applicants?
The double anonymous proposal review process may require some changes, particularly for new applicants, in the way they write their proposals. The Information for Applicants and Anonymisation Guide documents describe these changes and aid in the proposal preparation. The changes are mostly in the style, structure, and grammar used in describing the work.

While not a lot of work, it may not be as simple as resubmitting previous versions of the same manuscript. If you are re-applying, the previous version of your application will be available as read-only for you to use as a reference - https://my.nci.org.au/mancini/ncmas/2021/.

Please pay particular attention to the inclusion of career interruptions. This information will be used to create an anonymised track-record for your team that will be available to the NCMAC as part of their review. 

4.6    What changes have been made to the review process?
For the mandatory review phase, the reviewers will not know the identity of the applicants.

The first stage of review is mandatory for all reviewers. The NCMAC members are encouraged to focus on the scientific merit of the Proposal and the technical justification describing the use of HPC/D resource to address the science proposed as described in the Computational Details. The reviewers will also see an anonymised set of track-record metrics to help determine the capacity of a team to conduct the proposed research.

A second stage will be optional for NCMAC members. If a reviewer decides they need more information, they will be able to reveal the identifiable track record of a team. They will also be able to see the previous use of NCMAS allocated resources if applicable, or previous HPC experience at other facilities/through other schemes, which will contain identifiable details of previous HPC/D use and research outcomes. 

4.7    How do the reviewers assess the applicant's responsible use of the HPC/D resource, or likelihood of scientific return?
As with all prior NCMAS reviews, the NCMAC must use their expert judgement to determine whether each proposal would result in the proper use of the HPC/D resource and a scientific return on the project.

Each proposal will still have a facility technical assessment. 

We encourage applicants to take extra care to sufficiently justify the technical requirements of the project, such that the facility staff can review and appropriately report on feasibility. 

4.8    How can we be sure that accepted NCMAS proposals are actually feasible if the NCMAC can't assess the team's past experience?
As part of the review of applications, NCMAS facilities provide a technical assessment of projects. 

Technical staff from the facilities are highly experienced and will flag potential technical challenges in proposals. 

It is the responsibility of the NCMAS Custodians to ensure that the community has equal opportunity for the use of HPC/D resources, regardless of previous experience.

4.9    As a reviewer, how can I be sure that the applicants are being ethical when discussing their expertise and/or access to other facilities? What if we allocate time to the wrong people?
First, the NCMAC committee members review applications after facility technical review has occurred.

The NCMAC makes recommendations for allocations based on the merit of the science proposed. 

The Chair and Deputy review all allocations, and will have access to all identifying details of the applicants for use in this review, including names and affiliations.

4.10    How will this affect ECR Proposals?
All applications will be subject to anonymisation – the Proposal and Computational Details need to be written in an anonymous third-person (A3P) style. 

ECR applications will need to follow the same anonymisation to be compliant.   

4.11    When will anonymisation be implemented?
Now.
Anonymisation will be required for NCMAS 2022 – the call for applications for use in the 2022 calendar year – assessed by the NCMAC in late 2021.

4.12    What will happen to proposals that are not sufficiently anonymised?
Anonymisation of the Proposal and Computational Details is mandatory for compliance, and applicants must follow the guidelines laid out in the Information for Applicants. If an application is found to quite obviously and blatantly disregard the anonymising guidelines, the application will be deemed noncompliant and will be withdrawn from further consideration.

To support applicants in making this change, an optional A3P compliance check is available. Draft Proposal and Computational Details can be submitted before 7:00pm AEST (5:00pm AWST) 13 September 2021 for this review, and feedback on A3P compliance will be provided. 

This is not considered final submission, and if suggested changes are not made to the draft, the application may be found noncompliant. 

If an applicant submits Proposal and Computational Details documents that do not comply with anonymous third-person style, the application will be judged non-compliant.

4.13    I've followed the guidelines, but my work is so niche (or my methods so unique) that I fear reviewers may be able to determine my identity. Will I be flagged as non-compliant?
If applicants follow the guidelines for A3P style, their application will be compliant with anonymisation. 

It is not necessary to "water down" or obscure your science, methods, or tools, it is simply your responsibility to write about them in the third-person, in a way that does not intentionally identify yourself. 

4.14    How will the process deal with Conflicts of Interest?
In some respects, the reviewer COI with a given proposal are a bit simpler. If the reviewers do not know who the applicants are, how can they be conflicted?

As always, reviewers can (and should) identify issues not identified by our system, personal conflicts.

NCMAC members will not be assigned their own proposal for review.

4.15    Does this change make it harder to be awarded HPC/D resource?
No. NCMAS is allocating approximately the same amount of resource as in the previous cycle.

If the number of submitted proposals were to rise, this would affect the oversubscription rate, but this has always been the case. The same is true if the requested allocation amount increases across individual applications, and therefore the total requested amount could increase.

This process is not about making it harder for some people or easier for others to get HPC/D resources; it is about ensuring that the best proposals are selected.

Your best chance of being awarded HPC/D resource is the same it has always been: think of a great idea, and write a great proposal.

4.16    Instructions within the form state not to use author names in the Proposal and Computational Details, but also state that referencing is essential. Where do references go if I can't use names in the text?
The intention of the statement about the author name is to write application documents such that they cannot be matched to an identifiable author/applicant.

It is fine to write "As Doe et al. (2018) showed..." within the text as long as the writing is such that it is not obvious that the author of the application is Doe.

Please read through the Anonymisation Guide – this will demonstrate how referencing, with names, should look.

Please also make use of the optional compliance check before 7:00pm AEST (5:00pm AWST) 13 September 2021 if you would like feedback on your application's A3P compliance.

A reference list can still appear at the end of your document and it is acceptable for the references in this list to include names, including the application author's name. This is not seen as identifiable as a new project in the same field of science might have a similar reference list.