Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Saving LARGE app resources causes exception due to auditing. ("Data too long for column 'NewValue' at row 1") #829

Open
benanhalt opened this issue May 19, 2021 · 4 comments · May be fixed by #4889
Open

Saving LARGE app resources causes exception due to auditing. ("Data too long for column 'NewValue' at row 1") #829

benanhalt opened this issue May 19, 2021 · 4 comments · May be fixed by #4889
Assignees
@melton-jason
Labels
1 - Bug Incorrect behavior of the product 2 - Database/Schema Issues that are related to the underlying database and schema
Projects
Administration
Milestone
Grant's issue list

Comments

@benanhalt
Copy link
Contributor

Issue #632 was fixed by truncating field values when saving them to the audit log. But the issue reoccurred when trying to save changes to a form that involved adding a unicode character. I suspect the truncation which is done in Python is counting the characters while the MySQL field limit is on the number of bytes. Since a unicode character can consist of multiple bytes the truncated value can still exceed the field size.
@benanhalt benanhalt added the 1 - Bug Incorrect behavior of the product label May 19, 2021
@benanhalt
Copy link
Contributor Author

Here is the form that couldn't be saved.
badform.xml.txt

@maxpatiiuk maxpatiiuk added this to Unsorted in Users & Permissions via automation Sep 14, 2021
@maxpatiiuk maxpatiiuk removed this from Unsorted in Users & Permissions Apr 22, 2022
@maxpatiiuk maxpatiiuk added this to Unsorted in Administration via automation Apr 22, 2022
@grantfitzsimmons grantfitzsimmons added the 2 - Database/Schema Issues that are related to the underlying database and schema label Aug 8, 2023
@grantfitzsimmons
Copy link
Member

grantfitzsimmons commented Jan 29, 2024 •
edited

Still an issue. Reported by MCNB when editing large forms.

Workaround:

  1. Delete the form app resource
  2. Create a new one
  3. Load the data
  4. Save (no error, since it doesn't save the data to the audit log)

Too_Big_invert_views.xml.zip

@grantfitzsimmons grantfitzsimmons added this to the Grant's issue list milestone Jan 29, 2024
@melton-jason melton-jason self-assigned this Jan 29, 2024
@grantfitzsimmons
Copy link
Member

grantfitzsimmons commented Feb 27, 2024 •
edited

I am trying to edit the collectionobject data entry form for Ichthyology and have been getting some strange errors here.

First I tried to increase the column span for the field in question like below:

But for some reason I get a 500 error message, whenever I tried to save:

image

Looking at the server logs it says the following:

 [Tue Feb 27 06:10:54.399608 2024] [wsgi:error] [pid 27767] [remote 10.175.116.37:54130]   File "/home/specify/currentversion/specify7/ve/lib64/python3.6/site-packages/MySQLdb/connections.py", line 276, in query
 [Tue Feb 27 06:10:54.399610 2024] [wsgi:error] [pid 27767] [remote 10.175.116.37:54130]     _mysql.connection.query(self, query)
 [Tue Feb 27 06:10:54.399613 2024] [wsgi:error] [pid 27767] [remote 10.175.116.37:54130] django.db.utils.DataError: (1406, "Data too long for column 'NewValue' at row 1")

This is my complete view file:
fish.views.xml (72.3 KB)

Specify 7 Version: v7.7.5

Reported by: Fedor at NHMD on Discourse

@grantfitzsimmons grantfitzsimmons changed the title Saving LARGE app resources causes exception due to auditing. (Again) Saving LARGE app resources causes exception due to auditing. ("Data too long for column 'NewValue' at row 1") Feb 27, 2024
@specifysoftware
Copy link

This issue has been mentioned on Specify Community Forum. There might be relevant details there:

https://discourse.specifysoftware.org/t/specify7-crashes-with-certain-edits-to-preparation-type-subview-def/1579/2

melton-jason added a commit that referenced this issue May 5, 2024
@melton-jason
Correctly truncat value fields for spauditlog entries
f13e6b2 
Fixes #829
@melton-jason melton-jason linked a pull request May 5, 2024 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@melton-jason melton-jason

Labels
1 - Bug Incorrect behavior of the product 2 - Database/Schema Issues that are related to the underlying database and schema
Projects
Administration
  
Unsorted
Back-End Backlog
Status: 📋 Backlog
Milestone
Grant's issue list
Development

Successfully merging a pull request may close this issue.

Truncate value fields for Audit Log entries specify/specify7
5 participants
@benanhalt @grantfitzsimmons @maxpatiiuk @melton-jason @specifysoftware