I think the caching implementation is broken #746

New issue

Closed

opened 2026-02-20 12:06:47 -05:00 by deekerman · 3 comments

deekerman commented

2026-02-20 12:06:47 -05:00

Owner

Originally created by @glent1 on GitHub (May 26, 2020).

I've been trying to debug an issue whereby when using UsenetBlackhole downloading, ScanWatchFolder.GetItems never finds the cached items from the last pass. I think the problem is that the Set method in Cached.cs schedules a deletion of the key for 5 minutes in the future, but if an entry with the same key gets subsequently added (which it does at each pass), there is now a rogue deletion queued for some point in the future.

So (I think) if any download takes longer than 5 minutes, it can't ever complete because every time a value is added to the cache, there is a deletion for it scheduled at some point in the next minute, so it definitely won't be there a minute later.

I've changed my local implementation of Set to this ...

        public void Set(string key, T value, TimeSpan? lifetime = null)
        {
            Ensure.That(key, () => key).IsNotNullOrWhiteSpace();
            _store[key] = new CacheItem(value, lifetime);

            if (lifetime != null)
            {
                System.Threading.Tasks.Task.Delay(lifetime.Value).ContinueWith(t =>
                {
                    if (_store.TryRemove(key, out var temp2) && !temp2.IsExpired())
                    {
                        _store.TryAdd(key, temp2);
                    }
                });
            }
        }

which I realise has a minor race condition in it, but I think it's tolerable. It seems to be working better for me.

Apologies if I've misunderstood what is going on, but this sort of timing issue is difficult to trace because the act of observation affects the results.

Originally created by @glent1 on GitHub (May 26, 2020). I've been trying to debug an issue whereby when using UsenetBlackhole downloading, ScanWatchFolder.GetItems never finds the cached items from the last pass. I think the problem is that the Set method in Cached.cs schedules a deletion of the key for 5 minutes in the future, but if an entry with the same key gets subsequently added (which it does at each pass), there is now a rogue deletion queued for some point in the future. So (I think) if any download takes longer than 5 minutes, it can't ever complete because every time a value is added to the cache, there is a deletion for it scheduled at some point in the next minute, so it definitely won't be there a minute later. I've changed my local implementation of Set to this ... ``` public void Set(string key, T value, TimeSpan? lifetime = null) { Ensure.That(key, () => key).IsNotNullOrWhiteSpace(); _store[key] = new CacheItem(value, lifetime); if (lifetime != null) { System.Threading.Tasks.Task.Delay(lifetime.Value).ContinueWith(t => { if (_store.TryRemove(key, out var temp2) && !temp2.IsExpired()) { _store.TryAdd(key, temp2); } }); } } ``` which I realise has a minor race condition in it, but I think it's tolerable. It seems to be working better for me. Apologies if I've misunderstood what is going on, but this sort of timing issue is difficult to trace because the act of observation affects the results.