Processing bat acoustic data in Kaleidoscope Pro

Select the appropriate survey type for detailed guidance on assigning automated species IDs to recordings, scrubbing noise files, attributing metadata to recordings, and exporting metadata in the NABat bulk upload template. 

Processing stationary acoustic data in Kaleidoscope Pro

1. Open Kaleidoscope and click "Browse" to select the input directory of the folder containing your .wav/.zc files. Select the box beside the type of input files (WAV/ZC).​ NOTE: Select a folder containing all data for a single detector deployment (all nights at a single site). This is the broadest selection for which all files will share the same metadata (site name, latitude, longitude, etc.).

2. Download the required NABat metadata forms here (NABat_Metadata_Form_Kaleidoscope_‌Stationary.xml for use with steps 1-9 below and NABat_Stationary_Meta_Output_Only_KPRO.xml for use with steps 12-18 below). Next, click the drop-down menu above the "Default Project Form" and select "Add or Replace a Project Form." Navigate to NABat_Metadata_Form_Kaleidoscope_‌Stationary.xml and select the file. The NABat metadata form will now load in the bottom of the INPUTS section (the left half of the Kaleidoscope window).

3. Complete all metadata fields for which you have data. Required fields include GRTS Cell ID or Latitude and Longitude, Location Name, Survey Start Time, Survey End Time, Software Type, and Species List. Some fields require specific formatting. Hover the pointer over a field for formatting tips and/or a list of accepted entries.

4. Once all required metadata fields (and any additional fields for which you have data) have been completed, select the desired output directory to save processed files by clicking "Browse" in the top right of the window.

5. Once the output directory has been selected, choose "Nightly" from the "Create subdirectories" drop-down menu to keep new files organized by detector night. Next, select the type of output files you wish to create, WAV or ZC (ZC files cannot be converted to WAV). In order to successfully export metadata, you must select WAV or ZC files in the OUTPUTS window and create copies of the original files during this step. These copies will contain embedded metadata that will be the source of the "meta" output spreadsheet you will upload to NABat. 

Users with zero crossing files that end in the .# extension rather than .zc should select the "8.3 file names" and "Use .zc instead of .??#" options below "ZC files." Otherwise, Kaleidoscope will remove the seconds from the filename, increasing the likelihood of duplicate filenames (resulting from distinct files recorded within the same minute) that will create errors when uploading metadata to the NABat database.

6. Next, click the "Auto ID for Bats" tab at the top of the window. Select "Bats of North America" from the "Classifiers" drop-down menu. Select "+ 1 More Accurate (Conservative)" from the drop-down menu at the top right and select your state/region from the drop-down menu below the list of species. Species can be manually added or removed based on specific knowledge of species assemblages in your area, but this must be reflected in your NABat species list. 

7. Click "Process Files" in the bottom right corner of the window.

8. When Kaleidoscope has finished processing, review the automated identifications and confirm or downgrade the Kaleidoscope ID (manual ID). For stationary surveys, NABat guidance calls for one manual call verification per species per night in order to establish presence-absence. 

9. If your detector model automatically embeds a date/time stamp into call files, these data should auto-populate and appear in the "Audio Recording Time" column of the Kaleidoscope "meta" output. If so, you may skip the renaming process. However, before uploading metadata to NABat, ensure that the "Audio Recording Time" column contains data. Otherwise, file names must adhere to NABat's naming format, and users should follow the guidance in step 10. If the "Audio Recording Time" column successfully auto-populates, you may skip to step 11.

10. To rename files, open the NABat File Renamer or download the R script for renaming acoustic files. If using the NABat File Renamer, navigate to the folder containing the files you just created with metadata and IDs (this will be a subfolder within the output folder from step 4). Rename all files to ensure they are consistent with NABat naming format and to remove the _000 appended by Kaleidoscope. NABat naming format: GRTSID_SITENAME_YYYYMMDD_HHMMSS.wav/zc. More detailed guidance on renaming acoustic files is available here

11. Repeat steps 1-10 for all deployments you wish to process. Once completed, steps 12-18 will allow users to simultaneously export metadata for all processed deployments into a single bulk upload CSV.

12. Return to the batch processing window, click "Browse" in the "Input directory" window, and select the parent folder containing all of your processed deployments (the new files you created with metadata and IDs in steps 1-9). Select a new output directory. If the box beside "WAV files" or "ZC files" is still checked, uncheck the box (you won't be creating new files this time, only a "meta" file containing NABat metadata).

13. Click the "Auto ID for Bats" tab and select "DISABLED" in the "CLASSIFIERS" drop-down menu.

14. Replace the NABat_Metadata_Form_Kaleidoscope_SurveyType.xml file from step 2 with NABat_SurveyType_Meta_Output_Only_KPRO.xml. The data entry window will no longer include NABat metadata fields, however, the form will still export the embedded metadata from steps 1-9.

15. Click "Process Files" in the bottom right corner of the window.

16. Navigate to the current output directory folder and open the "meta" file. Check to ensure all fields for which you entered data are filled and ensure one or both of the Auto Id/Manual Id columns are filled.

17. Rename the file with a descriptive and unique name and save (the NABat system will automatically overwrite duplicate file names with the most recent version, so it is critical that your file name is unique).

 

Processing mobile transect data in Kaleidoscope Pro

1. Open Kaleidoscope and click "Browse" to select the input directory of the folder containing your .wav/.zc files. Select the box beside the type of input files (WAV/ZC).​ NOTE: In order to embed and export metadata, mobile transects must be processed individually. Select a folder containing data for a single mobile transect. 

2. Download the required NABat metadata forms here (NABat_Metadata_Form_‌Kaleidoscope_Mobile.xml for use with steps 1-9 below and NABat_Mobile_Meta_Output_Only‌_KPRO.xml for use with steps 12-18 below). Next, click the drop-down menu above the "Default Project Form" and select "Add or Replace a Project Form." Navigate to NABat_Metadata_Form_‌Kaleidoscope_Mobile.xml and select the file. The NABat metadata form will now load in the bottom of the INPUTS section (the left half of the Kaleidoscope window).

3. Complete all metadata fields for which you have data. Required fields include Location Name, Survey Start Time, Survey End Time, Software Type, and Species List. Some fields require specific formatting. Hover the pointer over a field for formatting tips and/or a list of accepted entries. Spatial information for the route should be uploaded or drawn prior to completing these steps and uploading metadata. Guidance for providing spatial data for the route is available here. Ensure that the "Location Name" you provide with call metadata matches the name of the spatial object you upload or create.

• If your mobile transect passes through > 1 cell and you have a record of the X,Y location where each call was recorded (using an external GPS or a GPS enabled detector), leave the GRTS Cell ID field blank. The NABat system will use the X,Y locations to auto-assign the correct GRTS cell to each call. Otherwise, calls recorded outside the primary cell will produce a Latitude-Longitude/GRTS ID mismatch error. If Latitude and Longitude are embedded in your call files, the Kaleidoscope metadata form will extract those values and auto-populate the metadata output. If Latitude and Longitude values were downloaded from a GPS unit, see step 17.  ​

• If you are unable to provide the X,Y location where each file was recorded, the NABat system can estimate the location based on user-provided spatial information for the route, timestamps of each file, and the start/end times of the survey. In this case, it is critical that users provide a spatial object for the route prior to uploading metadata, ensure timestamps are provided for each file, provide start and end times of the route, and leave the GRTS Cell ID column of the metadata upload blank. 

4. Once all required metadata fields (and any additional fields for which you have data) have been completed, select the desired output directory to save processed files by clicking "Browse" in the top right of the window.

5. Next, choose "Nightly" from the "Create subdirectories" drop-down menu if you wish to keep new files organized by detector night. Select the type of output files you wish to create, WAV or ZC (ZC files cannot be converted to WAV). In order to successfully export metadata, you must select WAV or ZC files in the OUTPUTS window and create copies of the original files during this step. These copies will contain embedded metadata that will be the source of the "meta" output spreadsheet you will upload to NABat.

• Users with zero crossing files that end in the .# extension rather than .zc should select the "8.3 file names" and "Use .zc instead of .??#" options below "ZC files." Otherwise, Kaleidoscope will remove the seconds from the filename, increasing the likelihood of duplicate filenames (resulting from unique files recorded within the same minute) that will create errors when uploading metadata to the NABat database.  ​

6. Click the "Auto ID for Bats" tab at the top of the window. Select "Bats of North America" from the "Classifiers" drop-down menu and select your state/region from the drop-down menu below the list of species. Species can be manually added or removed based on specific knowledge of species assemblages in your area, but this must be reflected in your NABat species list

7. Click "Process Files" in the bottom right corner of the window.

8. When Kaleidoscope has finished processing, review the automated identifications and confirm or downgrade the Kaleidoscope ID (manual ID). For mobile transects, NABat guidance calls for manual verification of all call files in order to estimate abundance.

9. If your detector model automatically embeds a date/time stamp into recordings, these data should auto populate and appear in the "Audio Recording Time" column of the Kaleidoscope "meta" output. If so, you may skip the file renaming process. However, before uploading metadata to NABat, you should ensure that the "Audio Recording Time" column contains data. Otherwise, file names must adhere to the NABat naming format, and users should follow the guidance in step 10. If the "Audio Recording Time" column successfully auto populates, you may skip to step 11.

10. To rename files, open the NABat File Renamer or download the R script for renaming acoustic files. If using the NABat File Renamer, navigate to the folder containing the files you just created with metadata and IDs (this will be a subfolder within the output folder from step 4). Rename all files to ensure they are consistent with NABat naming format and to remove the _000 appended by Kaleidoscope.

NABat naming format: GRTSID_SITENAME_YYYYMMDD_HHMMSS.wav/zc. More detailed guidance on renaming acoustic files is available here

11. Repeat steps 1-10 for all transects you wish to process. Once completed, steps 12-24 will allow users to simultaneously export metadata for all processed transects into a single bulk upload CSV.

12. Return to the batch processing window, click "Browse" in the "Input directory" window, and select the parent folder containing all processed transects (the new files created with metadata and species IDs in steps 1-10). Select a new output directory. If the box beside "WAV files" or "ZC files" is still checked, unselect the box (you won't be creating new files this time, only a "meta.csv" file containing NABat metadata).

13. Click the "Auto ID for Bats" tab and select "DISABLED" in the "CLASSIFIERS" drop-down menu.

14. Replace the NABat_Metadata_Form_Kaleidoscope_Mobile.xml file from step 2 with NABat_Mobile_Meta_Output_Only_KPRO.xml. The data entry window will no longer include NABat metadata fields; however, the form will still export the embedded metadata from steps 1-9.

15. Click "Process Files" in the bottom right corner of the window.

16. Navigate to the current output directory folder and open the "meta" file. Check to ensure all fields for which you have entered data are filled and ensure the Auto Id/Manual Id columns are filled.

17. Users who saved latitude and longitude of recordings with an external GPS unit must manually add X,Y locations to the bulk metadata spreadsheet. To begin, download the locations file from your GPS unit. This is typically in the form of a text file with columns for filename, latitude, and longitude. Users with GPS enabled detectors that embed X,Y locations into call files (the latitude/longitude columns of your CSV will already contain data) can skip to step 24.

18. Import the GPS text file into Excel:

• Open a new spreadsheet. Click the "Data" tab and then select "Get Data," then "From File," and finally "From Text/CSV."

• Navigate to your GPS text file and click "Open."

• A dialogue box will appear with a preview of the data to be imported. The GPS text files are typically tab delimited. If your data are properly sorted into columns, click "Load." If not, use the "Delimiter" drop-down menu to select the appropriate delimiter and click "Load."​

19. When your data are loaded into an Excel spreadsheet, click the "Sort and Filter" button (under the Home tab) and select "Custom Sort."​

20. In the dialogue box, select the column containing your filenames to "Sort by."

21. Next, open the CSV containing your transect metadata. Follow steps 19-20 to sort the metadata document by the Audio Recording Name column.

22. The two Excel documents (metadata CSV and GPS spreadsheet) should now be sorted by filename. Double check to ensure that the filename columns match.

23. If the documents are sorted properly, you can now copy and paste the latitude and longitude columns from the GPS file into the metadata CSV. Select the latitude and longitude columns of the GPS document. Highlight the latitude and longitude columns, right click, and select "Copy." Next, select the Latitude cell of the first call for the deployment, right click, and select "Paste." The Latitude and Longitude columns should now be filled with the corresponding values.

24. Rename the metadata file with a descriptive and unique name and save (the NABat system will automatically overwrite duplicate file names with the most recent version, so it is critical that your file name is unique). 

25. The metadata file is now ready to be uploaded to your NABat project.