caching - Race-condition creating folder in Python -

I have a urllib2 caching module, which crashes oddly due to the following code:

  If not os.path.exists (self.cache_location): os.mkdir (self.cache_location)  

The problem is that until the second row is executed, the folder exists Maybe, and the error will be:

 File "... / cache.py", in line 103, __init__ os.mkdir (self.cache_location) OSError: [Error 17] File exists: ' / Tmp / examplecachedir / '

This is because the script was launched several times Or has been, there is no control of my third-party code.

The code (before trying to fix the bug) can be found

I can not use it, because it can be used by a randomly named directory () Resolve the situation, which will defeat the purpose of the cash.

T to simply abandon the error, as error error Errno 17 error should be raised if the name of the folder exists as a file (a different error), for example:

< Ex> $ touch blah $ python >>> imported OS >>> os.mkdir ("blah") traceback (most recent call final): file "" in OSERR, line1,: [error 17] file exists : 'Bla' >>> << pre>

I can not use the threading. Rollock As the code is called from several processes.

So, I tried writing a simple file-based lock (), but there is a problem in it: This lockfile creates a level above, so /tmp/example.lock For / tmp / example / , if you use / tmp / as a cache directory (such as /tmp.lock Tries to create.)

In short, I need to cache the urllib2 response to the disk. To do this, I have a known directory The need to reach (it is possible, if necessary), a Bhuproses secure it OS X, Linux and need to work on Windows.

Thoughts? The only alternative way I can think, instead of files, is to rewrite the cache module using the SQLite3 storage.

instead

  if not os.path.exists (self .cache_location): os.mkdir (self.cache_location)  

you can

  try: osirr: osmcards (save cash_lion): pass  

As you end up with the same functionality

_{Disclaimer: I do not know how this can be. By using}

---

SQLite3 ,

If you choose "many" "To do the work, then , concurrently inserting and filtering, it is a good idea to use SQLite3 , because it does not add much complexity on simple files (argued It can be that it removes complexity).

---

Reading your questions (and comments) I can understand your problem better.

What is the probability that file can create a similar race situation?

If it is too small, then osource: pass

  if not os.path.isfile (self.cache_location): try Do: os.makedirs (self.cache_location) P> In addition, reading your code, I will change  
  else: # Our target dior is already a file, or different error, #default Relay! OSERR (E)  
 
 to 
 
  else: # Our goal is already a file from Dior, or increase different error, increase # relay error! Raise  
 
 As it really is what you want, the dragon rereise the exact same exception  (just nitpicking) . 
 
 
 One more thing can be used for you (just like Unix).