Supports maximum text length

[tianchiguaixia]

Regarding spacy_llm, what is the maximum text length it supports? If the text length in a single instance is very long, exceeding the limit of OpenAI, how does it handle this?

[rmitsch]

Hi @tianchiguaixia, currently spacy-llm doesn't do anything about overly long texts. We are currently finishing a feature though that will use a map-reduction approach to handling texts that are too long for models' context windows. In a nutshell, docs will be split in as many shards as necessary, each shard will be processed by the LLM independently, and finally all shards will be merged back into a single Doc instance.

This will be included in v1.0.0 of spacy-llm, which we hope to release in late December or early January.