Model builder checklist for aquatic biomonitoring database

CABIN Commitment

Two or three members of the Canadian Aquatic Biomonitoring Network (CABIN) Science Team and the appropriate CABIN regional lead will review any model submitted directly to the Science Team or through the National CABIN Team. The team members will review the information provided for consistency with the CABIN method, accuracy in the application of the reference condition approach, and statistical correctness.  Sufficient information must be provided for the reviewers to understand the decisions made through each step of the model development process. There must also be sufficient detail for practitioners to repeat the analysis. The reviewers will discuss the model via conference call within 4-6 weeks of receiving the model. Questions, comments, or recommendations will be provided back to the originator. Model upload to CABIN will be dependent on the review and discussed with the originator at that time.

Required information for model review

Model builders may provide information describing the model in any format (i.e. report submitted to contracting authority, journal publication). The information listed below should be provided in sufficient detail. Additionally, a standard Model Supporting Documentation Template must supplied with necessary model details for upload to CABIN (see below). The model template will be provided on the CABIN website in French and English for all CABIN users wishing to apply the model.

1. Study Design and Site Selection (also included in Model Template for website)
   1. Purpose of the model
   2. Geographical scope
   3. Temporal Scope
   4. Reference site stratification and site selection criteria/method
   5. Map
2. Data download and preparation (Are the data comparable and accurate?)
   1. Description of the CABIN studies and reference data used
   2. Steps taken to ensure data compatibility and accuracy (e.g. checks for unverified taxa, pelagic/terrestrial taxa, differences in sample processing/collection)
3. Data Description (What does the data look like?)
   1. Biological data
   2. Habitat data
4. Model development (How did you arrive at the final model?)
   1. Classification decisions including data transformations and exclusion of outliers
   2. List of Candidate predictor variables
   3. Discriminant Function Analysis decisions and supplementary analyses to arrive at the final model
   4. Summary of the final model (biological and habitat for each reference group)
5. Model performance evaluation (How did you test the model?)
   1. Classification and Cross-validation tables
   2. Results of known test sites, validation data or simpacted data
   3. Data gaps and inappropriate uses for model if applicable
   4. In the case of model updates/revisions, comparison to previous assessments and explanation for any differences

Information required for model upload to the CABIN database

Following the review and acceptance of the model by the CABIN Science Team, the following information is required for CABIN upload preferably in a single Excel Workbook. This information should be provided to the regional CABIN lead.

1. List of reference site codes with CABIN study name, date, sample number, CABIN assigned SampleID as they are extracted from the CABIN database.
2. Reference Group assignment for each site
3. Predictor variables as they are named and extracted in the CABIN database
4. DFA Classification Table
5. List of taxa that were not included in classification and assessment (if applicable)
6. Results of a few test sites (i.e. probability of group membership and ordination results) to ensure the functionality of the model once uploaded to CABIN.
7. Model Supporting Documentation Template completed for the CABIN website for CABIN users