Operation Error : Database is Locked.

[talk2jaydip]

Describe bug

While using yfinance from docker container, throwing Error saying Operation Error: Database is locked
ticke size is 100 symobls at time

# Define a function that takes a list of tickers and returns a DataFrame
def download_data(ticker=None):
    try:
        time.sleep(2)
        data = yf.download (tickers=ticker,group_by="Ticker", period="60d", interval="1h",proxy=None)
        
        return data

Simple code that reproduces your problem

give above

Debug log

AS shown above

Bad data proof

No response

yfinance version

latest

Python version

3.7

Operating system

ANY

[ValueRaider]

Try main branch #1080, it has relevant changes that should fix #1675.

If you confirm fix, I'll send out release asap

[talk2jaydip]

@ValueRaider :

Tried out this inside docker:
pip install git+https://github.com/ranaroussi/yfinance.git@main

[talk2jaydip]

FYI:

    # Create a pool of processes with the number of CPU cores available
    with concurrent.futures.ProcessPoolExecutor(max_workers=mp.cpu_count()) as executor:
        for c in itertools.islice(tasks,mp.cpu_count()-1):
            c = [stock for stock in c if stock not in failed_list_yf]
            futures.update({executor.submit(  download_data,list(c)  ): c[0]})

[ValueRaider]

So you're running multiple processes, not just threads?

[talk2jaydip]

YES... to make process other signal based on latest value. otherwise would take long time.
Issue is while running with Thread/Process PollExecutors

[ValueRaider]

yf.download in a loop = ‼️ Maybe rethink why do this. Maybe you need https://pypi.org/project/yfinance-cache/

[talk2jaydip]

Not in a loop, in simple terms sending 100 symbols at a time, so with multiprocessing sending 8 req at time so that can be downloaded parallelly.

Thanks for sharing yf-cache, i will look into it.

[ValueRaider]

multiprocessing sending 8 req at time

‼️ download is already parallel.