how to do cloud computing with php
in [General]
|
Prev: confirmation of what functions open_basedir will restrict?
Next: [PHP]Zip and text files generated are corrupted
From: Lester Caine on 24 Mar 2010 08:22 Rene Veerman wrote: > Hi.. > > As a way to take a few steps back from the kinda heated "when will php > grow up and support threading" thread, i'm requesting you people list > how you scale from 1 server to many servers; what's called cloud > computing. > > In particular, i'm interested in how to set up an application that > deals with great amounts of input from many 3rd-party servers, and say > a million concurrent viewers who need html calculated from those input > streams. > > So this goes beyond 1 mysql server, and beyond 1 php server. > > Let's hear it, coz quite frankly i have my doubts about php's ability > to scale to cloud computing. Multiple PHP boxes with local databases carrying the slowly changing data, only accessing the main database 'cloud' for real time data. Many financial institutions around the world use this arrangement without any problems. The data is replicated around the network and locally served up for each branch of the tree. You don't need all one million users looking to one point on the network? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php
From: Stuart Dallas on 24 Mar 2010 08:41 On 24 Mar 2010, at 11:55, Rene Veerman wrote: > As a way to take a few steps back from the kinda heated "when will php > grow up and support threading" thread, i'm requesting you people list > how you scale from 1 server to many servers; what's called cloud > computing. > > In particular, i'm interested in how to set up an application that > deals with great amounts of input from many 3rd-party servers, and say > a million concurrent viewers who need html calculated from those input > streams. > > So this goes beyond 1 mysql server, and beyond 1 php server. > > Let's hear it, coz quite frankly i have my doubts about php's ability > to scale to cloud computing. The way you're using the term cloud computing differs from the generally accepted definition, which is "delivering hosted services over the Internet". What you're talking about is scaling an application beyond a single server which is a different, albeit usually related, beast. There is no single or "right" way to implement a scalable web application but there are some key principles. The main principle that enables maintainable scalable systems is layers. If you build your application in a layered fashion, such that no layer knows how the other layers work, you can modify any of the layers in any way you need to when scalability issues arise. Your hardware setup will depend on the application and usage patterns, but some combination of load balancers, application servers and database servers is pretty normal. You may also find servers dedicated to caching and offline processing useful to achieve your goals. Architecture-wise you'll want to make extensive use of caching to lighten the load on the database, and you'll want to make sure that you perform no processing during a request that could wait to be processed offline. The main site I maintain has two load balancers at the front running nginx, three application servers running FastCGI PHP processes, two database servers in a master/slave arrangement, a separate static server and two offline processing servers. The application servers all have memcached instances for caching purposes and each runs its own instance of Sphinx to localise the search load. That setup, combined with a shared-nothing approach to the application design means I have a highly scalable system. To add capacity I simple need to add servers in whichever group is having issues. Due to the nature of the application I'm yet to come close to having issues with the number of concurrent writes to the database, but when I do I already plan to add another master server such that I have a master/master/slave setup. Unless you do start getting the sort of traffic that Facebook are blessed with you will be able to get a long way with this type of arrangement. And with a modular, layered approach to the application design I don't see any reason why I can't handle any scalability issue that's likely to come along with a minimum of fuss and cost. The key think to remember is that scalability is not the same as performance. They're barely even related. What, specifically, makes you doubt PHP's ability to scale? -Stuart -- http://stut.net/
From: Yousif Masoud on 24 Mar 2010 09:13 On Wed, Mar 24, 2010 at 11:55 AM, Rene Veerman <rene7705(a)gmail.com> wrote: > Hi.. > > As a way to take a few steps back from the kinda heated "when will php > grow up and support threading" thread, i'm requesting you people list > how you scale from 1 server to many servers; what's called cloud > computing. > Interesting definition of cloud computing. There is a book called "Building Scalable Web Sites" by Cal Henderson. http://books.google.co.uk/books?id=wIWU94zKEtYC&dq=Building+Scalable+Web+Sites&printsec=frontcover&source=bn&hl=en&ei=zg2qS7m1G9WO_AbL7_x1&sa=X&oi=book_result&ct=result&resnum=4&ved=0CBcQ6AEwAw#v=onepage&q=&f=false I recommend you take a look at Chapter 9: Scaling Web Applications, particularly the following sections: The Scaling Myth (particularly the subsection "What is Scalability?") Scaling PHP I quote "As PHP has gained more acceptance as a serious language, there's still been a significant camp of people claiming that it can't scale. Clearly, this isn't the case." page 212. It would be very helpful if you tell us what you've read, tried or considered?
From: Rene Veerman on 24 Mar 2010 15:02
On Wed, Mar 24, 2010 at 3:13 PM, Yousif Masoud <yousif.masoud(a)gmail.com> wrote: > On Wed, Mar 24, 2010 at 11:55 AM, Rene Veerman <rene7705(a)gmail.com> wrote: >> >> Hi.. >> >> As a way to take a few steps back from the kinda heated "when will php >> grow up and support threading" thread, i'm requesting you people list >> how you scale from 1 server to many servers; what's called cloud >> computing. > > Interesting definition of cloud computing. > > There is a book called "Building Scalable Web Sites" by Cal Henderson. > > http://books.google.co.uk/books?id=wIWU94zKEtYC&dq=Building+Scalable+Web+Sites&printsec=frontcover&source=bn&hl=en&ei=zg2qS7m1G9WO_AbL7_x1&sa=X&oi=book_result&ct=result&resnum=4&ved=0CBcQ6AEwAw#v=onepage&q=&f=false > > I recommend you take a look at Chapter 9: Scaling Web Applications, > particularly the following sections: > > The Scaling Myth (particularly the subsection "What is Scalability?") > > Scaling PHP > I quote "As PHP has gained more acceptance as a serious language, there's > still been a significant camp of people claiming that it can't scale. > Clearly, this isn't the case." page 212. ok, i'm gonna take a look at that book then. > > It would be very helpful if you tell us what you've read, tried or > considered? ok. i'm knowledgeable about coding for 1 to 2 machines; 1 php, 1 (my)sql. i can cache output/results at various levels for re-use. my main concern atm is my own cms (50-100k lines of my own); it's graphics-heavy, does fairly complicated db based logic, and if it ever is to be used for a site like facebook, it'll get large dataflows that have to be distributed over the servers used to generate html and accessoiries for end-users. i've built a layer into it that caches the output of oft-used pages (like articles and their comments). but adding many comments / minute to an article would result in quite a bit of overhead, to update the html for that page and distribute it (fast enough) to the relevant servers. i'm worried about php's single-threaded nature; each request has to fetch html updated in the last few seconds, or generate it from a list of comments. that's also a big query from a big table for every end-user.. :( i'd rather keep them comments for an article in shared memory..... |