Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing CSV without schema is not supported despite the documentation says otherwise #114

Open
adamsiemion opened this issue Nov 6, 2018 · 7 comments
Labels

Comments

@adamsiemion
Copy link

README says "But even if you do not know (or care) about column names you can read/write CSV documents". I think it is incorrect. It is impossible to write CSV without defining a schema first. Below code (in my example) will return Unrecognized column 'name': known columns: ]:

new CsvMapper()
     .enable(CsvParser.Feature.WRAP_AS_ARRAY)
     .writeValueAsString(users);

This is because of the following code in CsvGenerator:

CsvSchema.Column col = _schema.column(name, _nextColumnByName+1);
if (col == null) {
	if (isEnabled(JsonGenerator.Feature.IGNORE_UNKNOWN)) {
		_skipValue = true;
		_nextColumnByName = -1;
		return;
	}
	// not a low-level error, so:
	_reportMappingError("Unrecognized column '"+name+"': known columns: "+_schema.getColumnDesc());
}

Would it be possible to add support for writing CSV without defining a schema first?

@cowtowncoder
Copy link
Member

Interesting. Yes, you are right, currently only schema-less reading is supported, although in theory it should be possible to also support writing of arrays/Lists.

But whether it would be possible to expose such functionality via general databinding API is bit more difficult question. I guess the way it should work would be to allow writing of List/array values through writeValues() method of ObjectWriter, at high level.

One big problem would be to figure out if CsvGenerator could be modified easily to accept what looks like an Array, when it expects Object (to represent a single row).

Now. All of above might not be what you are looking for. It depends on what users is. If contents are POJOs, or Map, schema would definitely be needed.
But you would typically create one with something like:

CsvMapper.schemaFor(User.class);

@adamsiemion
Copy link
Author

@cowtowncoder thank you for a prompt reply. I have described in details my requirements here: https://stackoverflow.com/questions/53174072/automatically-serialize-rest-controllers-to-csv-instead-of-json-in-spring-boot

@cowtowncoder
Copy link
Member

Ah! So this would sort of belong to jackson-jaxrs-csv, if such was defined. I agree, it'd be nice to have a declarative way to indicate use of generated Schema. In fact we have:

FasterXML/jackson-jaxrs-providers#21

which would work for JAX-RS. Not sure if that'd help with Spring Boot (I forget how providers work there, Spring vs JAX-RS).

@Anchal2989
Copy link

I too have similar issue. Raised it here:
https://stackoverflow.com/questions/68949432/unable-to-generate-csv-from-avro-file-in-java
com.fasterxml.jackson.databind.JsonMappingException: [no message for java.lang.NullPointerException]

adamsiemion, kindly suggest if you already found a solution to this.

@cowtowncoder
Copy link
Member

cowtowncoder commented Aug 30, 2021

@Anchal2989 I don't think your problem is the same as you are defining CsvSchema; NPEs can result from many things.
However it seems likely that the way you are constructing ObjectWriter might prevent use of that schema. I'll add a note on SO question.

For anyone else following this: for SO question, the important part is to construct and use a SequenceWriter if writing rows one by one, instead of trying to create one ObjectWriter for each row.

@Anchal2989
Copy link

Hi @cowtowncoder,
Thanks for suggestion. I have tried all the possible ways to write the csv using csv writer/csvmapper/sequencewriter/objectwriter.
But unable to identify the issue why it causes NPE.
So moved the library to apache-csv and it works now.
I will wait if there are going to be any upgrades on this library.
If you have any sample kindly provide I will try to work on that.

@cowtowncoder
Copy link
Member

@Anchal2989 Sorry to hear you were not able to figure out a working way. I plan to write a blog some time this month to follow

https://cowtowncoder.medium.com/reading-csv-with-jackson-c4e74a15ddc1

but this time talk about writing, using SequenceWriter. Your use case can be made to work but it is not well documented and unfortunately project README does not show it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants