Working with a temporary file
in [Lisp]
|
From: Zach Beane on 11 Feb 2010 11:07 I need to work with a fresh, unique file that is deleted automatically when I'm done with it. Here are the functions I came up with, I'd love to hear any feedback. (defparameter *alphabet* (concatenate 'string "abcdefghijklmnopqrstuvwxyz" "0123456789" "ABCDEFGHIJKLMNOPQRSTUVWXYZ")) (defun random-string (length) "Return a random string with LENGTH characters." (let ((string (make-string length))) (map-into string (lambda (char) (declare (ignore char)) (aref *alphabet* (random (length *alphabet*)))) string))) (defun call-with-temporary-open-file (template fun &rest open-args &key element-type external-format) "Call FUN with two arguments: an open output stream and a file name. When it returns, the file is deleted. TEMPLATE should be a pathname that can be used as a basis for the temporary file's location." (declare (ignorable element-type external-format)) (flet ((new-name () (make-pathname :name (concatenate 'string (pathname-name template) "-" (random-string 8)) :defaults template))) (let (try stream) (tagbody :retry (setf try (new-name)) (unwind-protect (progn (setf stream (apply #'open try :if-exists nil :direction :output open-args)) (unless stream (go :retry)) (funcall fun stream try)) (when stream (close stream) (ignore-errors (delete-file try)))))))) ;Zach
From: Madhu on 11 Feb 2010 12:21 * Zach Beane <87r5orycwq.fsf(a)hangup.portland.xach.com> : Wrote on Thu, 11 Feb 2010 11:07:01 -0500: | I need to work with a fresh, unique file that is deleted automatically | when I'm done with it. Here are the functions I came up with, I'd | love to hear any feedback. | | (defun call-with-temporary-open-file (template fun &rest open-args | &key element-type external-format) | "Call FUN with two arguments: an open output stream and a file | name. When it returns, the file is deleted. TEMPLATE should be a | pathname that can be used as a basis for the temporary file's | location." | (declare (ignorable element-type external-format)) | (flet ((new-name () | (make-pathname :name (concatenate 'string | (pathname-name template) | "-" | (random-string 8)) | :defaults template))) | (let (try stream) | (tagbody | :retry | (setf try (new-name)) | (unwind-protect | (progn | (setf stream (apply #'open try | :if-exists nil | :direction :output | open-args)) | (unless stream | (go :retry)) | (funcall fun stream try)) | (when stream | (close stream) | (ignore-errors (delete-file try)))))))) I'd prefer a slightly different level of abstraction - Split the job into two parts 1) the creation of a unique temporary file and 2) its use by the programmer via standard WITH-OPEN-FILE/OPEN macros. This would punt the complexity introduced in massaging OPEN-ARGS, etc., to `where it belongs' The idiom I'd use is (let (file) (unwind-protect (with-open-file (stream (setq file (HCL:MAKE-TEMP-FILE)) :direction :io ...) ;; use stream ) (when file (delete-file file)))) (On Lispworks, There is an [undocumented?] function HCL:MAKE-TEMP-FILE) -- Madhu
From: Zach Beane on 11 Feb 2010 12:43 Madhu <enometh(a)meer.net> writes: > I'd prefer a slightly different level of abstraction - Split the job > into two parts 1) the creation of a unique temporary file and 2) its use > by the programmer via standard WITH-OPEN-FILE/OPEN macros. This would > punt the complexity introduced in massaging OPEN-ARGS, etc., to `where > it belongs' > > The idiom I'd use is > > (let (file) > (unwind-protect (with-open-file (stream (setq file (HCL:MAKE-TEMP-FILE)) > :direction :io ...) > ;; use stream > ) > (when file (delete-file file)))) It seems like this approach suffers from a race condition; I aimed to avoid that. With SBCL on Unix, :if-exists :error and :if-exists nil both eventually use the O_EXCL flag to open(), which guarantees (modulo NFS and the like) the stream is opened on a fresh file. I'd like to know if hcl:make-temp-file makes that a non-issue, somehow. Zach
From: Madhu on 11 Feb 2010 13:05 * Zach Beane <87k4ujy8fu.fsf(a)hangup.portland.xach.com> : Wrote on Thu, 11 Feb 2010 12:43:33 -0500: | Madhu <enometh(a)meer.net> writes: | |> I'd prefer a slightly different level of abstraction - Split the job |> into two parts 1) the creation of a unique temporary file and 2) its |> use by the programmer via standard WITH-OPEN-FILE/OPEN macros. This |> would punt the complexity introduced in massaging OPEN-ARGS, etc., to |> `where it belongs' |> |> The idiom I'd use is |> |> (let (file) |> (unwind-protect (with-open-file (stream (setq file (HCL:MAKE-TEMP-FILE)) |> :direction :io ...) |> ;; use stream |> ) |> (when file (delete-file file)))) | | It seems like this approach suffers from a race condition; I aimed to | avoid that. With SBCL on Unix, :if-exists :error and :if-exists nil both | eventually use the O_EXCL flag to open(), which guarantees (modulo NFS | and the like) the stream is opened on a fresh file. I'm not sure I understand what race contition you are seeing here, because the call to open(2) should NOT use the O_EXCL flag (i.e. create a fresh file) if you pass :IF-DOES-NOT-EXIST :ERROR, If it did that, it would be a bug in the implementation IMO. Note in the sketch above that the call to WITH-OPEN-FILE could or rather SHOULD be called with :IF-DOES-NOT-EXIST :ERROR, and the assumption is that WITH-OPEN-FILE opens an existing file which the MAKE-TEMP-FILE created. Only we (the callers of MAKE-TEMP-FILE) are responsible for deleting the created file, and no other file will be created with that name until we have deleted it. | I'd like to know if hcl:make-temp-file makes that a non-issue, somehow. If you split the job into the 2 pieces, MAKE-TEMP-FILE just needs to call creat(2), and it can possibly do this without going through CL:OPEN -- Madhu
From: Zach Beane on 11 Feb 2010 14:18
Madhu <enometh(a)meer.net> writes: > |> (let (file) > |> (unwind-protect (with-open-file (stream (setq file (HCL:MAKE-TEMP-FILE)) > |> :direction :io ...) > |> ;; use stream > |> ) > |> (when file (delete-file file)))) > | > | It seems like this approach suffers from a race condition; I aimed to > | avoid that. With SBCL on Unix, :if-exists :error and :if-exists nil both > | eventually use the O_EXCL flag to open(), which guarantees (modulo NFS > | and the like) the stream is opened on a fresh file. > > I'm not sure I understand what race contition you are seeing here, I'm referring to http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.html which says: Once you create the file atomically, you must alway use the returned file descriptor (or file stream, if created from the file descriptor using routines like fdopen()). You must never re-open the file, or use any operations that use the filename as a parameter - always use the file descriptor or associated stream. Otherwise, the tmpwatch race issues noted above will cause problems. > because the call to open(2) should NOT use the O_EXCL flag (i.e. create > a fresh file) if you pass :IF-DOES-NOT-EXIST :ERROR, If it did that, it > would be a bug in the implementation IMO. Right, when I mentioned O_EXCL, I was referring to my version, which uses ":if-exists nil", not :if-does-not-exist <anything>. > If you split the job into the 2 pieces, MAKE-TEMP-FILE just needs to > call creat(2), and it can possibly do this without going through CL:OPEN Is there a CL operation that corresponds closely to creat(2) like CL:OPEN with :if-exists :error-or-nil corresponds closely to open(2) with O_EXCL? Zach |