Parallel Common-Lisp with at least 64 processors?
in [Lisp]
|
From: Jon Harrop on 20 Jun 2008 09:32 Robert Maas, http://tinyurl.com/uh3t wrote: > The whole point of distributing my tasks over 64 processors is to > try to get it to run 64 times as fast as if running on a single > processor, to allow a couple orders of magnitude more data to be > processed within a reasonable time. If you want your programs to run at a reasonable speed you should not be using Lisp in the first place. So your best bet is probably to rewrite your code in a modern language and benefit from their enormous performance improvements. -- Dr Jon D Harrop, Flying Frog Consultancy http://www.ffconsultancy.com/products/?u
From: Ariel on 20 Jun 2008 11:18 On Fri, 20 Jun 2008 14:32:50 +0100 Jon Harrop <jon(a)ffconsultancy.com> wrote: > Robert Maas, http://tinyurl.com/uh3t wrote: > > The whole point of distributing my tasks over 64 processors is to > > try to get it to run 64 times as fast as if running on a single > > processor, to allow a couple orders of magnitude more data to be > > processed within a reasonable time. > > If you want your programs to run at a reasonable speed you should not be > using Lisp in the first place. So your best bet is probably to rewrite your > code in a modern language and benefit from their enormous performance > improvements. I thought using a modern compiler for Lisp would allow it to perform as well as any other standard modern day high level language? (Or so says Paul Graham.) Is this a false statement?
From: Edi Weitz on 20 Jun 2008 11:51 On Fri, 20 Jun 2008 08:18:55 -0700, Ariel <no(a)mail.poo> wrote: > I thought using a modern compiler for Lisp would allow it to perform > as well as any other standard modern day high level language? (Or > so says Paul Graham.) Is this a false statement? The statement is correct. The person who posted the mis-information you replied to is a well-known troll who tries everything (including blatant lies) to sell his books and magazines. Search the archives of this newsgroup for other postings of him. Edi. -- Lisp is not dead, it just smells funny. Real email: (replace (subseq "spamtrap(a)agharta.de" 5) "edi")
From: Jon Harrop on 20 Jun 2008 16:19 Ariel wrote: > On Fri, 20 Jun 2008 14:32:50 +0100 > Jon Harrop <jon(a)ffconsultancy.com> wrote: >> Robert Maas, http://tinyurl.com/uh3t wrote: >> > The whole point of distributing my tasks over 64 processors is to >> > try to get it to run 64 times as fast as if running on a single >> > processor, to allow a couple orders of magnitude more data to be >> > processed within a reasonable time. >> >> If you want your programs to run at a reasonable speed you should not be >> using Lisp in the first place. So your best bet is probably to rewrite >> your code in a modern language and benefit from their enormous >> performance improvements. > > I thought using a modern compiler for Lisp would allow it to perform as > well as any other standard modern day high level language? (Or so says > Paul Graham.) Is this a false statement? That certainly is a false statement, yes. Even for trivial tasks, it can be vastly more difficult to write comparably efficient Lisp code. For example, this toy symbolic simplifier: http://www.lambdassociates.org/studies/study10.htm The OCaml code is not particularly efficient and OCaml itself is not particularly well-suited to this benchmark (MLton-compiled SML would be a lot faster): let rec ( +: ) f g = match f, g with | `Int n, `Int m -> `Int (n +/ m) | `Int (Int 0), e | e, `Int (Int 0) -> e | f, `Add(g, h) -> f +: g +: h | f, g -> `Add(f, g) let rec ( *: ) f g = match f, g with | `Int n, `Int m -> `Int (n */ m) | `Int (Int 0), e | e, `Int (Int 0) -> `Int (Int 0) | `Int (Int 1), e | e, `Int (Int 1) -> e | f, `Mul(g, h) -> f *: g *: h | f, g -> `Mul(f, g) let rec simplify = function | `Int _ | `Var _ as f -> f | `Add (f, g) -> simplify f +: simplify g | `Mul (f, g) -> simplify f *: simplify g The obvious Lisp is 7.5x slower than the unoptimized OCaml: (defun simplify (a) (if (atom a) a (destructuring-bind (op x y) a (let* ((f (simplify x)) (g (simplify y)) (nf (numberp f)) (ng (numberp g)) (+? (eq '+ op)) (*? (eq '* op))) (cond ((and +? nf ng) (+ f g)) ((and +? nf (zerop f)) g) ((and +? ng (zerop g)) f) ((and (listp g) (eq op (first g))) (destructuring-bind (op2 u v) g (simplify `(,op (,op ,f ,u) ,v)))) ((and *? nf ng) (* f g)) ((and *? (or (and nf (zerop f)) (and ng (zerop g)))) 0) ((and *? nf (= 1 f)) g) ((and *? ng (= 1 g)) f) (t `(,op ,f ,g))))))) If you openly advertise this as a programming challenge and get dozens of expert Lisp programmers to painstakingly optimize solutions to this trivial problem over a period of many weeks then the fastest implementation you get is still 1.7x slower than the unoptimized OCaml but, more importantly, it is a completely unmaintainable mess: (defun simplify-no-redundant-checks (xexpr) (if (atom xexpr) xexpr (let ((op (first xexpr)) (z (second xexpr)) (y (third xexpr))) (let* ((f (simplify-no-redundant-checks z)) (g (simplify-no-redundant-checks y)) (nf (numberp f)) (ng (numberp g))) (tagbody START (if (eq '+ op) (go OPTIMIZE-PLUS) (go TEST-MULTIPLY)) OPTIMIZE-PLUS (when (and nf ng) (return-from simplify-no-redundant-checks (+ f g))) TEST-PLUS-ZEROS (when (eql f 0) (return-from simplify-no-redundant-checks g)) (when (eql g 0) (return-from simplify-no-redundant-checks f)) (go REARRANGE-EXPR) TEST-MULTIPLY (unless (eq '* op) (go REARRANGE-EXPR)) OPTIMIZE-MULTIPLY (when (and nf ng) (return-from simplify-no-redundant-checks (* f g))) TEST-MULTIPLY-ZEROS-AND-ONES (when (or (eql f 0) (eql g 0)) (return-from simplify-no-redundant-checks 0)) (when (eql f 1) (return-from simplify-no-redundant-checks g)) (when (eql g 1) (return-from simplify-no-redundant-checks f)) REARRANGE-EXPR (when (and (listp g) (eq op (first g))) (let ((op2 (first g)) (u (second g)) (v (third g))) (declare (ignore op2)) (return-from simplify-no-redundant-checks (simplify-no-redundant-checks (list op (list op f u) v))))) MAYBE-CONS-EXPR (if (and (eq f z) (eq g y)) (return-from simplify-no-redundant-checks xexpr) (return-from simplify-no-redundant-checks (list op f g)))))))) Lisp's awful performance only gets worse as your programs get more complicated. For anything non-trivial, Lisp is inevitably incredibly slow. Eventually, all Lisp programmers end up Greenspunning features like optimizing pattern matchers over algebraic data types that are taken for granted in modern functional languages. As a dynamic language, Lisp cannot statically check even the most basic of constraints so programmers are forced to code in the debugger and waste their lives writing unit testing code. Fortunately, there are a wealth of much better modern functional programming languages out there for you to use, like SML, OCaml, Haskell, F# and Scala. They are not only many times faster than Lisp for real work but also much more expressive and concise with better tools and much larger communities of friendly users. If you are interested in earning a living then I strongly recommend taking a look at Scala and F# because they already have vastly wealthier markets than Lisp will ever have. Regards, -- Dr Jon D Harrop, Flying Frog Consultancy http://www.ffconsultancy.com/products/?u
From: Rainer Joswig on 20 Jun 2008 16:30
In article <kMGdnSwTlOs_jMHVnZ2dnUVZ8vGdnZ2d(a)posted.plusnet>, Jon Harrop <jon(a)ffconsultancy.com> wrote: > Ariel wrote: > > On Fri, 20 Jun 2008 14:32:50 +0100 > > Jon Harrop <jon(a)ffconsultancy.com> wrote: > >> Robert Maas, http://tinyurl.com/uh3t wrote: > >> > The whole point of distributing my tasks over 64 processors is to > >> > try to get it to run 64 times as fast as if running on a single > >> > processor, to allow a couple orders of magnitude more data to be > >> > processed within a reasonable time. > >> > >> If you want your programs to run at a reasonable speed you should not be > >> using Lisp in the first place. So your best bet is probably to rewrite > >> your code in a modern language and benefit from their enormous > >> performance improvements. > > > > I thought using a modern compiler for Lisp would allow it to perform as > > well as any other standard modern day high level language? (Or so says > > Paul Graham.) Is this a false statement? > > That certainly is a false statement, yes. Even for trivial tasks, it can be > vastly more difficult to write comparably efficient Lisp code. For example, > this toy symbolic simplifier: ___________________________ /| /| | | ||__|| | Please don't | / O O\__ feed | / \ the troll | / \ \ | / _ \ \ ---------------------- / |\____\ \ || / | | | |\____/ || / \|_|_|/ | __|| / / \ |____| || / | | /| | --| | | |// |____ --| * _ | |_|_|_| | \-/ *-- _--\ _ \ // | / _ \\ _ // | / * / \_ /- | - | | * ___ c_c_c_C/ \C_c_c_c____________ |