KeyError: 'exchangeTimezoneName' when accessing ticker history #1076

manoelmarques · 2022-10-05T23:33:23Z

Current yfinance 0.1.75 raises an exception when running the following:

import yfinance as yf


def _main():
    stock_data = yf.download(
        "AFN",
        start="2018-01-01",
        end="2018-12-31",
        group_by="ticker",
        threads=False
    )
    print(stock_data)


if __name__ == "__main__":
    _main()

  stock_data = yf.download(
  File "/Users/manoel/opt/anaconda3/envs/Qiskitenv/lib/python3.8/site-packages/yfinance/multi.py", line 120, in download
    data = _download_one(ticker, period=period, interval=interval,
  File "/Users/manoel/opt/anaconda3/envs/Qiskitenv/lib/python3.8/site-packages/yfinance/multi.py", line 205, in _download_one
    return Ticker(ticker).history(period=period, interval=interval,
  File "/Users/manoel/opt/anaconda3/envs/Qiskitenv/lib/python3.8/site-packages/yfinance/base.py", line 148, in history
    end = utils._parse_user_dt(end, self._get_ticker_tz())
  File "/Users/manoel/opt/anaconda3/envs/Qiskitenv/lib/python3.8/site-packages/yfinance/base.py", line 328, in _get_ticker_tz
    tkr_tz = self.info["exchangeTimezoneName"]
KeyError: 'exchangeTimezoneName'

Running the same sample against the previous release returns:

[*********************100%***********************]  1 of 1 completed
                 Open       High        Low      Close  Adj Close  Volume
Date                                                                     
2018-01-02  29.799999  29.799999  29.799999  29.799999  29.799999       0
2018-01-03  30.000000  30.000000  30.000000  30.000000  30.000000       0
2018-01-04  30.000000  30.000000  30.000000  30.000000  30.000000       0
2018-01-05  30.200001  30.200001  30.200001  30.200001  30.200001       0
2018-01-08  30.000000  30.000000  30.000000  30.000000  30.000000       0
2018-01-09  30.200001  30.200001  30.200001  30.200001  30.200001       0
2018-01-10  32.799999  32.799999  32.799999  32.799999  32.799999       0
2018-01-11  31.799999  31.799999  31.799999  31.799999  31.799999       0
2018-01-12  30.799999  30.799999  30.799999  30.799999  30.799999       0
2018-01-16  30.200001  30.200001  30.200001  30.200001  30.200001       0
2018-01-17  30.799999  30.799999  30.799999  30.799999  30.799999       0
2018-01-18  30.799999  30.799999  30.799999  30.799999  30.799999       0
2018-01-19  30.799999  30.799999  30.799999  30.799999  30.799999       0
2018-01-22  31.400000  31.400000  31.400000  31.400000  31.400000       0
2018-01-23  32.000000  32.000000  32.000000  32.000000  32.000000       0
2018-01-24  31.600000  31.600000  31.600000  31.600000  31.600000       0
2018-01-25  31.200001  31.200001  31.200001  31.200001  31.200001       0
2018-01-26  32.200001  32.200001  32.200001  32.200001  32.200001       0
2018-01-29  31.799999  31.799999  31.799999  31.799999  31.799999       0
2018-01-30  31.520000  31.520000  31.520000  31.520000  31.520000       0

If I run with threads=True it will raise the exception and hang.

The text was updated successfully, but these errors were encountered:

ValueRaider · 2022-10-06T01:47:00Z

What is "AFN"? I don't see it in Yahoo. And the old price data looks bad - 0 volume, low=high.

manoelmarques · 2022-10-06T14:18:18Z

The example above is actually from the London Stock Exchange. I found that using "AFN.L" it returns data and doesn't raise an exception .

So it seems the exception in the new release happens for invalid tickers. If I enter ticker "???" in the new release I get the key error exception. In the previous version there was no key error, just a printed message:

[*********************100%***********************]  1 of 1 completed

1 Failed download:
- ???: No data found for this date range, symbol may be delisted

ValueRaider · 2022-10-06T14:27:20Z

That's a good point, it should print a nice message. Will fix for next release.

felixlulz · 2022-10-06T18:27:30Z

hey, i got the same error. it used to print

BEFORE:

KGF.L
0
[100%] 1 of 1 completed
LAND.L
1
[100%] 1 of 1 completed
LGEN.L
2
[100%] 1 of 1 completed
LLOY.L
3
[100%] 1 of 1 completed
LSEG.L
4
[100%] 1 of 1 completed
MNG.L
5
[100%] 1 of 1 completed
MGGT.L
6
[100%**] 1 of 1 completed

1 Failed download:

???: No data found for this date range, symbol may be delisted

NOW:

raises error and hangs like this when i loop trough the ticker symbols. not fixable with try/except

KGF.L
0
[100%] 1 of 1 completed
LAND.L
1
[100%] 1 of 1 completed
LGEN.L
2
[100%] 1 of 1 completed
LLOY.L
3
[100%] 1 of 1 completed
LSEG.L
4
[100%] 1 of 1 completed
MNG.L
5
[100%] 1 of 1 completed
MGGT.L
6
Exception in thread Thread-38:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/dist-packages/multitasking/init.py", line 104, in _run_via_pool
return callee(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/yfinance/multi.py", line 193, in _download_one_threaded
keepna, timeout)
File "/usr/local/lib/python3.7/dist-packages/yfinance/multi.py", line 210, in _download_one
timeout=timeout)
File "/usr/local/lib/python3.7/dist-packages/yfinance/base.py", line 148, in history
end = utils._parse_user_dt(end, self._get_ticker_tz())
File "/usr/local/lib/python3.7/dist-packages/yfinance/base.py", line 328, in _get_ticker_tz
tkr_tz = self.info["exchangeTimezoneName"]
KeyError: 'exchangeTimezoneName'

hoping for a fix soon, thanks for your work and help

ValueRaider · 2022-10-08T10:25:15Z

Should be fixed now

ValueRaider · 2022-10-09T10:47:05Z

@c0mm0d0re What ticker is generating error? I am getting correct behaviour for both listed and delisted tickers.

ValueRaider · 2022-10-09T14:03:04Z

I'm not going to run every ticker in the universe and get my IP blocked/restricted. Just tell me what ticker failed for you.

Also quicker this way.

…id of multiple calls to self.info (get_info).

fredrik-corneliusson · 2022-10-23T20:48:01Z

I'm having the same issue (using the dev branch)
Looking at the code there is a safeguard that checks for the key exchangeTimezoneName in self.info before getting it.

if not 'exchangeTimezoneName' in self.info:
    return None
tz = self.info["exchangeTimezoneName"]

However as info is backed by a property function (defined in the subclass) and calls self.get_info (defined in base class) it probably does multiple requests and the self.info can differ.

    @property
    def info(self):
        return self.get_info()

I updated the code in my forked dev branch so the KeyError exception issue should not be triggerd.
fork

However, the fact that the code get different results from consecutive calls to self.get_info() is the root cause of this bug and not something my code change fixes

My Traceback:

Exception in thread Thread-1000:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File .virtualenvs/screener/lib/python3.7/site-packages/multitasking/__init__.py", line 104, in _run_via_pool
    return callee(*args, **kwargs)
  File "/yfinance-tz-cache/yfinance/multi.py", line 201, in _download_one_threaded
    keepna, timeout)
  File "/yfinance-tz-cache/yfinance/multi.py", line 218, in _download_one
    timeout=timeout)
  File "/yfinance-tz-cache/yfinance/base.py", line 157, in history
    tz = self._get_ticker_tz()
  File "/yfinance-tz-cache/yfinance/base.py", line 400, in _get_ticker_tz
    tz = self.info["exchangeTimezoneName"]
KeyError: 'exchangeTimezoneName'

ValueRaider · 2022-10-23T21:11:06Z

it probably does multiple requests and the self.info can differ.

It does make multiple requests but only the first does work. All subsequent requests will quickly return:

yfinance/yfinance/base.py

Line 362 in 34dfe94

return

Were you able to reliably produce 'different results from consecutive calls'?

fredrik-corneliusson · 2022-10-23T21:30:50Z

That depends:

yfinance/yfinance/base.py

Line 358 in 34dfe94

    
           if (self._info is None) or (self._sustainability is None) or (self._recommendations is None):

The get info does a lot of requests and if some of them fails in the first call that set self._info, self._sustainability or self._recommendations then all requests are redone if I read the code correctly.

fredrik-corneliusson · 2022-10-23T21:54:26Z

Have now tired to understand the code a bit more.
The get_info method only does one HTTP request but a lot of parsing of that request.
If the parsing of request response fails on _sustainability or _recommendations fails it will redo the HTTP request next time it is called.

ValueRaider · 2022-10-23T22:02:29Z

I understand now. Changing that if-statement from ors to ands seems best.

fredrik-corneliusson · 2022-10-23T22:56:09Z

I created a #1109 with my fix, it does however not use and in the if statment just a simplified version of the if statment. I can update that if you feel that change would be safe.

Fix for #1076

fredrik-corneliusson · 2022-10-24T10:03:08Z

@ValueRaider I looked at the json result we get from the endpoint that gets history ticker data
e.g https://query2.finance.yahoo.com/v8/finance/chart/TSLA
In that response there is already "exchangeTimezoneName" present.
Why do we get the timezone information from the scraping endpoint instead of from the ticker history response directly?
Using the scraping endpoint is probably slower and more error prone.

ValueRaider · 2022-10-24T10:31:48Z

At that point in the code, where start & end dates being processed, the price request hasn't been sent.

fredrik-corneliusson · 2022-10-24T11:26:43Z

Yes, but would it not be better to use that endpoint (giving a minimal period) to get the timezone?
That way any HTML scraping (and dependency on html not changing) would be avoided.
I did a quick comparison between the 2 endpoints:

Using query2 would give a speedup of 10 times.

And the extra query to ticker data could be avoided completely for longer ticker timeintervals (>1d) by asking for an extra day before and after and do the timezone filtering of the returned data.

ValueRaider · 2022-10-24T11:42:01Z

Good point. I'll push an update to dev

fredrik-corneliusson · 2022-10-24T12:15:21Z

If you are going to do multiple request to query2 to get the timezone maybe the code that does the request should be refactored in order to be reused.

I noted that this try/except clause effectively negates the explicit raising of runtime error to inform about yahoo is down.
https://github.com/ranaroussi/yfinance/blob/dev/yfinance/base.py#L211-L226

What is the desired behaviour? Code like this seems very ambiguous error prone to me.
In most cases silencing all exceptions is a bad idea, instead catch the ones you are expecting and handle them (logging or re-raising etc.) and in case of unexpected exception at least log it if you catch it.

ValueRaider · 2022-10-24T12:43:28Z

So that try/except predates me. Introduced by PR #907, maybe they forgot to remove the except - Ran has a habit of merging without reviewing.

Regarding timezone - I've pushed to fix/get-tz-performance. ~~Actual performance boost not as good as 10x because still requires json parsing. Maybe I'm missing an obvious optimisation?~~ Actually, isolating out the fetch the speedup is good.

ValueRaider · 2022-10-27T21:16:51Z

Can everyone update to 0.1.84 then report on the status of the bug? Is it solved?

fredrik-corneliusson · 2022-10-29T13:49:30Z

Seems solved for me.

manoelmarques mentioned this issue Oct 5, 2022

Update Yahoo Finance tutorial with popular companies qiskit-community/qiskit-finance#211

Merged

ValueRaider mentioned this issue Oct 7, 2022

Fix/info access when unlisted #1078

Merged

fredrik-corneliusson added a commit to fredrik-corneliusson/yfinance-tz-cache that referenced this issue Oct 23, 2022

Small cleanup to ease finding bug ranaroussi#1076. Begun by getting r…

d01d378

…id of multiple calls to self.info (get_info).

ValueRaider mentioned this issue Oct 23, 2022

Fix for #1076 #1109

Merged

ValueRaider added a commit that referenced this issue Oct 23, 2022

Merge pull request #1109 from fredrik-corneliusson/dev

cf795ea

Fix for #1076

This was referenced Oct 24, 2022

Improve performance of fetching Ticker timezone #1112

Merged

Yfinance Intermittently working #1111

Closed

ValueRaider added the Yahoo spam label Oct 27, 2022

ValueRaider closed this as completed Nov 10, 2022

ValueRaider removed the Yahoo spam label Jul 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KeyError: 'exchangeTimezoneName' when accessing ticker history #1076

KeyError: 'exchangeTimezoneName' when accessing ticker history #1076

manoelmarques commented Oct 5, 2022

ValueRaider commented Oct 6, 2022

manoelmarques commented Oct 6, 2022

ValueRaider commented Oct 6, 2022 •

edited

felixlulz commented Oct 6, 2022

ValueRaider commented Oct 8, 2022

ValueRaider commented Oct 9, 2022 •

edited

ValueRaider commented Oct 9, 2022 •

edited

fredrik-corneliusson commented Oct 23, 2022 •

edited

ValueRaider commented Oct 23, 2022 •

edited

fredrik-corneliusson commented Oct 23, 2022 •

edited

fredrik-corneliusson commented Oct 23, 2022

ValueRaider commented Oct 23, 2022

fredrik-corneliusson commented Oct 23, 2022

fredrik-corneliusson commented Oct 24, 2022

ValueRaider commented Oct 24, 2022

fredrik-corneliusson commented Oct 24, 2022 •

edited

ValueRaider commented Oct 24, 2022

fredrik-corneliusson commented Oct 24, 2022

ValueRaider commented Oct 24, 2022 •

edited

ValueRaider commented Oct 27, 2022

fredrik-corneliusson commented Oct 29, 2022

KeyError: 'exchangeTimezoneName' when accessing ticker history #1076

KeyError: 'exchangeTimezoneName' when accessing ticker history #1076

Comments

manoelmarques commented Oct 5, 2022

ValueRaider commented Oct 6, 2022

manoelmarques commented Oct 6, 2022

ValueRaider commented Oct 6, 2022 • edited

felixlulz commented Oct 6, 2022

ValueRaider commented Oct 8, 2022

ValueRaider commented Oct 9, 2022 • edited

ValueRaider commented Oct 9, 2022 • edited

fredrik-corneliusson commented Oct 23, 2022 • edited

ValueRaider commented Oct 23, 2022 • edited

fredrik-corneliusson commented Oct 23, 2022 • edited

fredrik-corneliusson commented Oct 23, 2022

ValueRaider commented Oct 23, 2022

fredrik-corneliusson commented Oct 23, 2022

fredrik-corneliusson commented Oct 24, 2022

ValueRaider commented Oct 24, 2022

fredrik-corneliusson commented Oct 24, 2022 • edited

ValueRaider commented Oct 24, 2022

fredrik-corneliusson commented Oct 24, 2022

ValueRaider commented Oct 24, 2022 • edited

ValueRaider commented Oct 27, 2022

fredrik-corneliusson commented Oct 29, 2022

ValueRaider commented Oct 6, 2022 •

edited

ValueRaider commented Oct 9, 2022 •

edited

ValueRaider commented Oct 9, 2022 •

edited

fredrik-corneliusson commented Oct 23, 2022 •

edited

ValueRaider commented Oct 23, 2022 •

edited

fredrik-corneliusson commented Oct 23, 2022 •

edited

fredrik-corneliusson commented Oct 24, 2022 •

edited

ValueRaider commented Oct 24, 2022 •

edited