Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Datamodel and Django Model definitions #4764

Open
wants to merge 40 commits into
base: production
Choose a base branch
from
Open

Conversation

acwhite211
Copy link
Member

@acwhite211 acwhite211 commented Apr 9, 2024

Fixes #4358 and #2813

This PR adds datamodel and Django model code definitions, instead of relying on reading the 'specify_datamodel.xml' file from Specify 6. So, the schema is now statically defined in our python Django code, instead of dynamically be created at runtime. There is also code that can generated the static python code from the 'specify_datamodel.xml', so we can regenerate the code again in the future easily if we need to. Unit tests have been created to ensure the equivalence between the old and new datamodels. There was another branch django-schema that had a lot of code from the development process, but I cleaned it up and simplified it into this branch. There were some problems with adding the sql alchemy schema model classes, this isn't a pressing matter so this is something we can do in the future.

Checklist

  • Self-review the PR after opening it to make sure the changes look good
    and self-explanatory (or properly documented)
  • Add automated tests
  • Add relevant issue to release milestone

Testing instructions

Unfortunately there isn't anything specific to test. I would say the best way to go about testing this from the UX side is to play around with forms and queries, and make sure no database table or field related errors occur. Also, for the sp7_only tables, test them out in the query builder.

sp7_only tables:

  • Spuserexternalid
  • Spattachmentdataset
  • UniquenessRule
  • UniquenessRule_Field
  • Message
  • Spmerging
  • UserPolicy
  • Role
  • LibraryRole
  • UserRole
  • RolePolicy
  • LibraryRolePolicy
  • Spdataset

PR Testing Checklist:

  • QB works on tables like Agent, Collecting Object, and Locality
  • QB works on the new sp7_only tables
  • The Database Schema page works without errors (UserTools->DatabaseSchema) localhost/specify/data-model

@acwhite211 acwhite211 added the 2 - Database/Schema Issues that are related to the underlying database and schema label Apr 9, 2024
@acwhite211 acwhite211 added this to the 7.9.6 milestone Apr 9, 2024
@acwhite211 acwhite211 self-assigned this Apr 9, 2024
@acwhite211 acwhite211 marked this pull request as draft April 9, 2024 17:21
Copy link
Collaborator

@lexiclevenger lexiclevenger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing instructions

Unfortunately there isn't anything specific to test. I would say the best way to go about testing this from the UX side is to play around with forms and queries, and make sure no database table or field related errors occur. Also, for the sp7_only tables, test them out in the query builder.

sp7_only tables:

  • Spuserexternalid
  • Spattachmentdataset
  • UniquenessRule
  • UniquenessRule_Field
  • Message
  • Spmerging
  • UserPolicy
  • Role
  • LibraryRole
  • UserRole
  • RolePolicy
  • LibraryRolePolicy
  • Spdataset

PR Testing Checklist:

  • QB works on tables like Agent, Collecting Object, and Locality
  • QB works on the new sp7_only tables
  • The Database Schema page works without errors (UserTools->DatabaseSchema) localhost/specify/data-model

Trying to open results from queries for the new tables results in a 404 error:

Screen.Recording.2024-06-05.at.3.16.57.PM.mov

'java.math.BigDecimal': 'models.DecimalField',
'java.lang.Double': 'models.FloatField', # This mapping wasn't in the original code
'java.lang.Boolean': 'models.BooleanField',
'text': 'models.TextField',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to shot it down. there is a discrepancy in how datamodel defines spappresourcedata. It is actually a BLOB but treated as a TextField in django, which required ugly workaround in record merging. Could you look at schema (in mariadb) and figure out more cases like that, and possibly resolve them?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2 - Database/Schema Issues that are related to the underlying database and schema
Projects
Status: Dev Attention Needed
Development

Successfully merging this pull request may close these issues.

Django Schema Independence
3 participants