Skip to content

Update Openspecimen to 6.3RC8 - #1

Open
ChristianMUG wants to merge 3519 commits into
bibbox:masterfrom
krishagni:master
Open

Update Openspecimen to 6.3RC8#1
ChristianMUG wants to merge 3519 commits into
bibbox:masterfrom
krishagni:master

Conversation

@ChristianMUG

Copy link
Copy Markdown

Getting the newest branch from openspecimen

…art IDs

in the sub-query that selects the carts eligible for read by the signed-in
user.
gracefully to prevent crashes during domain-to-interaction object conversion.
1) Switched from static table to expand/collapse accordion style table.
   Fixed the accordion table to handle all types of fields.

2) Displayed registration details when users click on one of the matched
   participant registration.

   Turned the table into clickable cards. When a matching card is clicked,
   a new card with participant registration details appear.

3) Displayed matching participant details using the dictionary configured
   for the registration CP.

4) Added ability to configure widgets within the matched participant details card.

   Implemented AQL backed tables.

   When no widgets are configured, the matched participant details card
   defaults to displaying collected specimens and aliquots count tables.

   Example configuration:

   {
     "name": "matching-participants",
     "data": {
       "detail-widgets": [
         {
           "title": "Primary Specimens",
           "aql": "select Specimen.type, Specimen.tissueSite, Specimen.spmnCollRecvDetails.collContainer, Specimen.availableQty where Specimen.lineage = \"New\" and Specimen.collectionStatus = \"Collected\" and Participant.id = :cprId",
           "params": {
             "dynamic": {
               "cprId": "cpr.id"
             }
           }
         },

         ...
       ]
     }
   }
As soon as the file was selected, the file uploader was unmounted.
However, the uploader runs the code assuming the file input is still
present in the view. As a result, the UI thread crashes with error
and no file is uploaded to the backend.
…tly parse

the multi-valued parameters cpShortTitle and dpShortTitle into an array.

For example
dpShortTile=A,B&dpShortTitle=C is translated to ["A,B", "C"]

Earlier it was translated to ["A", "B", "C"]
(Accidentally  committed the changes to master instead of VP_AI_EMBEDDINGS)

This reverts commit 22ef7f4.
…gning algorithm.

Problem: The logout request and the signature were encoded and sent as SAMLRequest.
SAMLRequest=<Logout Request + Signature on Logout Request>
Further, the URL also didn't specify the algorithm used to sign.

The required format is:
SAMLRequest=<Logout Request>&SigAlg=<Signing Algorithm>&Signature=<Covers both the Logout Request and signing algo>

Fixed the logout URL to conform to the above format.
Request managers can now be specified at the catalog or CP level.
… of all the

input objects identified by comma-separated names (labels, ppids, visit names, etc)

The problem with the earlier implementation was the names was fed into the query
but the output were limited to first 100 form records.

The fix is to divide the input names into chunks of 100 names. Iterate through each
chunk to query and fetch all records, append the records to the CSV file.

Care has been taken to ensure the looping does not break if any of the chunks
return no records.
… job.

Earlier approach: A single query computed collected, missed, not collected,
pending specimen counts for every participant in the database. The output
was subsequently used to create a new stats table.

While this approach worked well for databases with smaller number of participants
and specimens, it started failing on larger databases due to large transaction
size and connection timeouts.

Redesign:
In view of the above, the the job is redesigned to populate the stats in chunks.
By default, participants are divided in chunks of 10K. The job iterates through
each chunk to compute and insert the stats sequentially.

While this might not significantly reduce the execution time, it keeps the
transaction sizes in check and avoids connection timeouts.

Users (super admin) can fine tune the right chunk size for their database by
specifying a numeric value in the "Fixed Parameters" text box of the add/edit
job UI form.
specimen stats for every registered participant across all CPs.
1. Added many-to-one association of CP with CPG. Removed the intermediary
   OS_CP_GROUP_CPS table.

2. Fixed CP add/edit APIs to add / remove to / from group

3. Fixed CPG list API to return the CPs list in chunks / pages instead of
   all the CPs.

4. Added APIs to add / remove bulk CPs to / from CPG

5. Updated the CP APIs to update CP - CPG association in bulk using CSV
   import / export

6. Audit association of CP - CPG

7. Modified permissions: Users can add / remove CP to / from group provided
   they have CP update rights. This is different than earlier approach,
   which required users to have update rights on all group CPs.

8. Other access restrictions continue AS IS.

(At no point in time, all CPs of the group are materialised/fetched into
memory unless the selected page size is large)
Replaces the inline save button with a manual modal dialog to ensure
signature is saved. The modal persists until the user explicitly
selects 'Done' or 'Cancel'.

Why auto-save was not implemented?
Auto saving signatures introduces technical and reliability debt:
a) Stroke Detection: Hard to determine when a user is actually finished.
b) Concurrency: Out of sequence HTTP requests
c) Debouncing: Perceived lag and inconsistent state.

Underlying principle is always to keep things simple.
…rm records.

1) Send Survey: Added a button to the record overview that triggers an
   email/SMS link to participants for remote completion.

2) Survey Mode: Provides an UI toggle for users to enter form data using
   the survey interface.
… including the IDs.

This helps the exported JSON to be reusable across multiple DB instances as long as
the names match or remain consistent.
Add a new "User Roles (Detailed)" export while retaining the existing
"User Roles" export for backward compatibility.

Expand role assignments into individual (user, CP, site, role) rows:
1. (Site, CP): emit one row with the specific site and CP.

2. (All Sites, CP): emit one row for every active site of the CP belonging
   to the user's institute.

3. (Site, All CPs): emit one row for every active CP at the Site

4. (All Sites, All CPs): emit one row for every active CP-Site
   combination belonging to the user's institute.

Exclude

1. inactive CPs and sites

2. assignments with no matching expansion

Process selected users and CP-site mappings in batches. Use lightweight
database projections to avoid loading complete CP and site
object graphs into memory.
Allow PV, site, user, and storage-container fields to be configured as
either single-valued or multi-valued.

Preserve the selected cardinality after field creation

Support batched loading of saved selections, and display multiple values
as a comma-separated list.

Extend lookup APIs, form schemas, query UI, and import/export handling
to work with multiple lookup values supplied using their display names.

Dynamic Extensions library: #fed1a8a9e388ea29859b34846acc76f09bcfaed9
…bal promotion

Form designers can now create and manage form-specific dropdowns without leaving
the Form Designer. These dropdowns can be reused within their owning form,
cloned independently, and exported or imported along with the form.

The Dropdown Manager now supports finding form-specific dropdowns and promoting
one or more of them to globally available dropdowns. Forms are automatically
updated after promotion.
before widening its width.

This is an Oracle-specific change. Function-based indexes create pseudo
columns hidden behind the scenes. These pseudo columns depend on the
underlying columns. Altering the width or data type of the underlying
columns affect the indexes.

Fix this by dropping the index before widening the column and recreating
it afterwards.
1. Add pvRadioButton and pvCheckbox controls using centrally managed
   permissible values instead of inline options.

2. Support designer previews, options-per-row layouts, form-scoped PVs,
   queries, bulk import/export, and conversion to PV dropdowns.

3. Warn users when more than 25 options exist while preserving selected
   values outside the displayed option set.

4. Refactor common PV control and factory logic into shared abstractions
   and ensuring the existing pvField type functions and backward
   compatibility are not compromised.
…d Checkboxes.

Hide the newer PV-based Radio and Checkboxes.

Rename the PV based "Dropdown" to "Permissible Value"

Reason: Having many PV based fields means the query module would have to join
with the PV table for each field appearing in the query either a filter, or
select field, or grouping category. There's a hard limit of 61 joins imposed by
MySQL. In view of this, we are taking one step back to ensure users are not
impacted by this and solve the query problem before moving to PV based fields.
Allow radio buttons, checkboxes, single-select dropdowns, and multi-select
dropdowns to be converted to PV-based dropdown fields.

Users can select an existing PV attribute or create a new form-scoped
attribute using the field's existing options.

Migrate existing data by renaming the value column, creating a new column
for PV IDs, and resolving stored option values to the corresponding PV IDs.

Restore schema changes and delete newly created PV attributes when the
conversion fails.
Set the existing user ID on the updated user instance to prevent a
TransientPropertyValueException while adding the auto login audit log.

Partial updates were unaffected because UserFactory already assigns the
existing user ID to the updated instance.
…rotocol

Report that the collection protocol ID or short title may be missing or
invalid when the domain (CPR/Visit/Specimen) object has no resolved
collection protocol.
Reject checkout when the container is already checked out or has a
blocked checkout position.

This prevents duplicate transfer events and retains the container's
valid checkout state.
1. Repair missing object type and ID values from v12.2.x upgrades

2. Associate legacy form files immediately after MigrateFormFiles
   populates them.
Rename the reserved "file" alias to "dfile" in the Liquibase and Java migration
SQL to prevent the ORA-02012 missing USING keyword error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants