Have you read Cloud Databases – I love it! – Part-I?. This blog is written in such a way that you can understand even if you don’t read part-1. However, reading part-1 first will help you even better. Your call!.
I am going to continue with farming as an example for continuity and also for easy understanding.
My dad is my hero!.
I know what you think next, What about Mom?. She is the heroine!. What else?.. Few people who wanted take a shot at me is thinking of asking what about My wife?. She is above all!..(You can’t catch me.!).
Didn’t I tell you (I did not but checking you) we had farms in three different places?. 1. my dad inherited, 2. dad’s self made and 3. courtesy from my mom’s parents!. (my Dad got two jackpots, My mom and also some land. I got none. When I asked, my wife told me I am lucky to get her and I already won mega millions and powerball together!.).
In a nutshell you know my Dad had similar roadblocks like SMP systems had. (Part-1)
I am going to tell you a story of how did he overcome those obstacles and then explaining MPP is much easier!. You will also get to see the glimpse of why my Dad is my hero!.
He was a born leader who has never hesitated to take risk anytime in his life for good and he always thought differently than many others. So he looked at different kinds of farming we do and decided to pick the one that is more critical and most beneficial for us instead of trying to chew all at the same time. We had more coconut trees than others which had more potential for income than other types of farming. He chose to concentrate on coconut farming..So he has hired X-number of small but faster people who can climb quickly and dedicated them to each farm (dedicated CPU), hired Y-number of strong people who can carry large number of coconuts in bigger baskets (dedicated DISKS) and large tractors (COMMON NETWORK) to move coconuts between farms based on the demand, mix and match sizes (resource optimization) of coconuts etc). Wow!. We were able to witness the jobs done faster, generated more income and able to sell the coconuts sooner even during the peak harvest season since we had dedicated workers in each location (did NOT share them at all).
But one issue still persisted and couple more issues started adding up.
- When there are not enough rain or slow season of the harvest, we still had to maintain all resources (humans, animals and equipments) even though they were barely used. This did not go away even with new approach
- Dedicated resources means more expensive too but with more coutcome, my dad was fine.
- We started getting more demand for coconuts, but we were limited with amount of land (the size of the box) we had even after we have expanded our farm with 100% coconut. I wished we could grab our neighbors land!. Even when we had more demands we could not sell more since we have maxed out on land. (maxed out on the MPP box)
- Since land was limited (the MPP Box), my dad thought about buying another land (another box), but dropped the idea due to steep curve between cost vs benefits. He made a wise decision of not doing it so.
Go back and read ONLY highlighted texts with the context of computers. You will see MPP.
Now let us get little more technical but still I would like to aim for semi or non-technologist but not limited to. So what is MPP?
Massively Parallel Processing (MPP)
Some smart one’s came up with an idea of something called as Massively Parallel Processing (MPP) aka shared nothing architecture to overcome the limitations of SMP. Think of MPP as many mini computers inside a big computer and each minicomputer will have its own CPU, its own disk and memory. They don’t share anything at all. Like our kids!.
How did they achieve this shared nothing architecture?. Let us take a simple example. Not exactly the real technical greatness but for people to understand easily.
When you store a file, it is splited into multiple small files and a portion is stored in each computer. Now when you want to read that file it will be read “n” times faster because all mini computers works in parallel and uses its OWN memory and its portion of the file. For non-techno people, instead of one person sorting all your goods, you have say 10 people each sorting portion of your goods and achieving the productivity of “about” 10 times faster. Wow!. Yes Wow!
So what is MPP?.
Massively Parallel Processing (MPP) is the coordinated processing of a single task by multiple processors, each processor using its own OS and memory and communicating with each other using some form of messaging interface.
Now all your queries ran much faster and met all your business demand. Bingo and no more sleepless nights?. Correct?. Yes to an extent but not fully. It also came with a price such as you have to know the best way of splitting the file based on most used access pattern. (A long story about data distribution and zone maps). Also we have to have additional workload management where low priority requests don’t take the resources when high priority requests comes in. The price of these computers became very expensive too because they found a way to solve your pain!. They also hit the wall of how many mini computers they can fit inside a box?. (Horizontal Scalability)
Lastly, think about a scenario where you bought these expensive boxes with associated overheads, but your business does not need all the processing power for 80% of the time. Meaning you have a sunken cost and maintenance cost even though you want the power as little as needed and not the entire infrastructure during non-peak hours of your business. For non-technologist, you built 10 bedroom with huge event hall for once a year get-together and not able to use them for rest of the year. In addition to huge onetime sunken cost, still you have to manage and maintain the house and the county is not going to make your property tax low just because you don’t use the house all the times!. Merciless counties!. You get it.
The technology owners were very happy till the explosion and massive growth of data with all digitization happened in this decade. They could not add unlimited number of mini computers inside a physical box, faced difficulties in connecting all together efficiently thru high bandwidth network interfaces and suffered to provide faster response times when more data is added. So they eventually became slow and unable scale either horizontally (unable to add unlimited mini computers) or vertically (adding more processing power and memory for each mini computers). Since these computers need to know in advance how the data need to be distributed based on most accessing pattern, it was not able to give better throughout when people asked a question that was not based on the access pattern used to divide the data.
SMP vs MPP for technologists
Before I move into next section, I wanted to let you know that I have nothing but great regards on these technologies and I can’t thank enough.
So how do we solve the issues in-front of us?. We needed a technology better than MPP. Do you all agree?. Yes we all did and the help came in. A “BIG” help.
Time to turn the page to part 3 of the blog!.
Disclaimer
- While every caution has been taken to provide readers with most accurate information and honest analysis, please use your discretion before taking any decisions based on the information in this blog. Author(s) will not compensate you in any way whatsoever if you ever happen to suffer a loss / inconvenience / damage because of / while making use of information in this blog
- The views, thoughts, and opinions expressed in the blog belong solely to the author(s), and not necessarily to the author’s employer, organization, committee or other group or individual
- While the Information contained in this blog has been presented with all due care, author(s) warrant or represent that the Information is free from errors or omission
- Reference to any particular technology does not imply any endorsement, non-endorsement, support or commercial gain by the author(s). Author(s) are not compensated by any vendor in any shape or form
- Author(s) have no liability for the accuracy of the information and cannot be held liable for any third-party claims or losses of any damage
- If you like it or dislike, post your comments. Former motivates us to share more to our community and later helps us to learn from you but neither is going to stop us!
- Pardon us for any grammar errors and spelling mistakes which we know there are many!
Manix Views 