Refactor Validation and Create RollCall module - #56
Conversation
Move following functions: validate_csv_extension validate_output_directory validate_filename_uniqueness handle_missing_filenames setup_expected_columns
egrace479
left a comment
egrace479
left a comment
There was a problem hiding this comment.
Found a holdover from prior implementation method.
Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>
- Change tests - Include check_existing_images
This reverts commit ff930b0.
codedbyishika
changed the title
egrace479
left a comment
egrace479
left a comment
There was a problem hiding this comment.
I haven't had a chance to check the test changes, but some efficiency can be added and I have some questions in implementation of new features that rely on consideration of the motivation. Mainly the new duplicate check that should have an option to ignore for the purposes of still downloading, but then what is the intended response to duplicates; if the ignore flag is passed, should it be skipped or at least provide the duplicates in a usable format so the user can do something with the information?
Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>
Update newline in print statements. Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>
egrace479
left a comment
egrace479
left a comment
There was a problem hiding this comment.
Can you go through and check the Exceptions that are called (see for instance, these two)? We should be using the src/cautiousrobot/exceptions.py file we already have. This can be done in a separate PR to this branch.
| missing_cols.append(col) | ||
| if len(missing_cols) > 0: | ||
| raise Exception(f"The CSV at {csv_path} is missing column(s): {missing_cols}, defined as {[expected_cols[col] for col in missing_cols]}.") | ||
| raise Exception(f"The CSV at {csv_path} is missing column(s): {missing_cols}, defined as {[expected_cols[col] for col in missing_cols]}.") # noqa: TRY002 |
There was a problem hiding this comment.
We should have a MissingColumnsException defined in src/cautiousrobot/exceptions.py.
| try: | ||
| data_df = process_csv(csv_path, expected_cols) | ||
| except Exception as missing_cols: | ||
| except Exception as missing_cols: # noqa: BLE001 | ||
| sys.exit(f"{missing_cols} Please adjust inputs and try again.") |
There was a problem hiding this comment.
This try...catch seems to be duplicating the exception already raised by process_csv that should exit.
Move following functions from main.py to Validation.py + Change AGENTS.md:
validate_csv_extension
validate_output_directoryThis function was removed, as it should have been in #43.validate_filename_uniqueness
handle_missing_filenames
setup_expected_columns
RollCall Module
Added above functionalities and check_existing_images to the rollcall module which prechecks all the conditions before downloading.
Non Interactive Missing File Handling
Duplicate Checksums
Summary