SQLiteCache is not hit on Windows with Anthropic tools

[BrandonStudio]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

claude = ChatAnthropic(
	model_name="claude-3-sonnet-20240229",
	anthropic_api_url=claudeApiUrl,
	anthropic_api_key=claudeApiKey,
	default_headers={"anthropic-beta": "tools-2024-04-04"},
	cache=SQLiteCache("cache/claude-3-sonnet-20240229.db"),
)

prompt = ChatPromptTemplate.from_messages([
	("system", system),
	("user", "{input}")
])

claude_tools = claude.bind_tools([Result])

parsed = prompt | claude_tools.with_retry(retry_if_exception_type=(RateLimitError,))

class BatchCallback(BaseCallbackHandler):
	def __init__(self, total: int):
		super().__init__()
		self.count = 0
		self.progress_bar = tqdm(total=total)

	def on_llm_end(self, response: LLMResult, *, run_id: UUID, parent_run_id: UUID | None = None, **kwargs: Any) -> Any:
		self.count += 1
		self.progress_bar.update(1)

	def __enter__(self):
		self.progress_bar.__enter__()
		return self
	
	def __exit__(self, exc_type, exc_value, exc_traceback):
		self.progress_bar.__exit__(exc_type, exc_value, exc_traceback)

	def __del__(self):
		self.progress_bar.__del__()

df = pd.read_excel(path, sheet_name='Sheet1')
df = df[['title', 'full_text']]
dd = df.to_dict(orient='records')

with BatchCallback(len(dd)) as cb:
	dc = parsed.batch([{"input": {
		"title": d["title"],
		"full_text": d["full_text"]
	}} for d in dd], {"max_concurrency": 2, "callbacks": [cb]})

Error Message and Stack Trace (if applicable)

No response

Description

As the code provided, I used SQLiteCache to store the result of Anthropic with tools.
Run the same code twice, it is expected that

1. the second time is much faster than the first time
2. I do not see any log in Cloudflare AI Gateway during the second run
3. the SQLite database file is not written (i.e. file size is not increased) during the second run

But, what happend is that

1. the second time was as slow as the first time (no speed promotion)
2. I saw logs in Cloudflare AI Gateway during the second run
3. the SQLIte database file size was doubled after the second run

I have investegated the database file, found the prompt field of correspond records are completely same, and the llm field of correspond records have only one difference:

-"repr": "<langchain_community.cache.SQLiteCache object at 0x00000234A5107730>"
+"repr": "<langchain_community.cache.SQLiteCache object at 0x00000195B402AA40>"

I have another Linux with langchain-community==0.0.36 and Anthropic without tools, which could hit the cache.

System Info

> pip show langchain-community
Name: langchain-community
Version: 0.0.38
Summary: Community contributed LangChain integrations.
Home-page: https://github.com/langchain-ai/langchain
Author:
Author-email:
License: MIT
Location: d:\program files\python3\lib\site-packages
Requires: aiohttp, dataclasses-json, langchain-core, langsmith, numpy, PyYAML, requests, SQLAlchemy, tenacity
Required-by: langchain