Surprisingly though, several vendors went down the path of making their data warehousing offerings fatter and fatter. Greenplum is a good example. Prior to Sun’s acquisition by Oracle, they were heavily pushing systems based on the Sun Thumper, a 48-disk-heavy 4U box that can store up to 100TBs/box. I was quite familiar with that box as it partly came out of a startup called Kealia that my Stanford advisor, David Cheriton, and Sun co-founder Andy Bechtolsheim had founded and then sold to Sun in 2004. I kept wondering, though, what a 50TB/CPU configuration has to do with data analytics.

After long deliberation I came to the conclusion that it has nothing to do with it. There were two reasons why people were interested in this configuration. First, there were some use cases that required “near-line storage”, a term that’s used to describe a data repository whose major purpose is to store data but also allows for basic & infrequent data access. In that respect, Greenplum’s software on top of the Sun Thumpers represented a cheap storage solution that offered basic data access and was very useful for applications where processing or analytics was not the main focus.

The second reason for the interest, though, is a tendency to drive DW projects towards an absolute low per-TB price to reduce costs. Experienced folks will recognize that such an approach leads to disaster, because – as mentioned above – analytics is more than just Terabytes. Perfectly low per-TB price using fat storage looks great on glossy paper but in reality it’s no good because nobody’s analytical problems are that simple.

The point here is that analytics have more to do with processing rather than storage. It requires a fair number of balanced servers (thus good scalability & fault tolerance), CPU cycles, networking bandwidth, smart & efficient algorithms, fair amounts of memory to avoid thrashing etc. It’s also about how much processing can it be done by SQL, and how much of your analytics need to use next-generation interfaces like MapReduce or pre-packaged in-database analytical engines. In the new decade in which we’re embarking, solving business problems like fraud, market segmentation & targeting, financial optimization, etc., require much more than just cheap, overweight storage.

So going to the EMC/Greenplum news, I think such an acquisition makes sense, but in a specific way. It will lead to systems that live between storage and data warehousing, systems able to store data and also give the ability to retrieve it on an occasional basis or if the analysis required is trivial. But the problems Aster is excited about are those of advanced in-database analytics for rich, ad hoc querying, delivered through a full application environment inside a MPP database. It’s these problems that we see as opportunities to not only cut IT costs but also provide tremendous competitive advantages to our customers. And on that front, we promise to continue innovating and pushing the limits of technology as much as possible.

Syndicated via RSS From: http://www.asterdata.com/blog

Today there in research and development in major software organizations from IBM to Google to Apple about the crowdsourcing model of innovation. Infact Apple had adopted to this model and the success of iPhone is a very shining example. So did Google with Android platform, today there are scores of applications for this platform all over the world.

Software as we know it and Business as we knew it will change in the next five years to a total model of crowd driven innovation and growth model. Your Business is going Social, adopt to the new ways of the new world.

Syndicated via RSS From: http://www.b-eye-network.in/blogs/krishnan/

There are several things to understand in this move

1. Majir mover advantage for a commercial cloud provision – apart from Amazon
2. EMC establishing its presence as a player in the DWBI and Analytics space
3. The fact that BI in the cloud is a reality
4. The new platform for DWBI and Analytics may move out of SQL
5. DWBI as we know it will change

As we start moving from a SQL intensive DWBI to a non-SQL intensive BI, we will start looking at these kinds of platforms for a new alternative. Good news for all technology enthusiasts and Analytical BI users.

Another fact is now the DW Appliances Space has very few pure-play vendors – Netezza vs. AsterData is certainly the next battle

Syndicated via RSS From: http://www.b-eye-network.in/blogs/krishnan/

So what do you do with all that memory? There are two classes of use cases that are emerging today. First is the case where you need to increase concurrent access to data with reduced latency. Tools like memcached offer in-memory caching that, used properly, can vastly improve latency and concurrency for large-scale OLTP applications like websites. Also the nice thing with object caching is that it scales well in a distributed way and people have build TB-level caches. Memory-only OLTP databases have started to emerge, such as VoltDB. And memory is used implicitly as a very important caching layer in open-source key-value products like Voldemort. We should only expect memory to play a more and more important role here.

The second way to use memory is to gain “processing flexibility” when doing analytics. The idea is to throw your data into memory (however much it fits, of course) without spending much time thinking how to do that or what queries you’ll need to run. Because memory is so fast, most simple queries will be executed at interactive times and also concurrency is handled well. European upstart QlikView exploits this fact to offer a memory-only BI solution which provides simple and fast BI reporting. The downside is its applicability to only 10s of GBs of data as Curt Monash notes.

By exploiting an MPP shared-nothing architecture, Aster Data has production clusters with TBs of total memory. Our software takes advantage of memory in two ways: first, it uses caching aggressively to ensure the most relevant data stays in memory; and when data is in memory, processing is much faster and more flexible. Secondly, MapReduce is a great way to utilize memory as it provides full flexibility to the programmer to use memory-focused data structures for data processing. In addition, Aster Data’s SQL-MapReduce provides tools to the user to encourage the development of memory-only MapReduce applications.

However, one shouldn’t fall into the trap of thinking that all analytics will be in-memory anytime soon. While memory is down to $30/GB, disk manufacturers have been busy increasing platter density and dropping their price to less than $0.06/GB. Given that the amount of data in the world grows faster than Moore’s law and memory, there will always be more data to be stored and analyzed than what fits into any amount of memory that an enterprise can use. In fact, most big data applications will have data sets that do not fit into memory because, while tools like memcached worry only about the present (e.g. current Facebook users), analytics need to worry about the past, as well – and that means much more data. So a multi-layer architecture will be the only cost-effective way of analyzing large amounts of data for some time.

One shouldn’t be discussing memory without mentioning solid-state disk products (like Aster Data partner company Fusion-io). SSDs are likely to make the surprise here given that their per-GB price is falling faster than disks (being a solid-state product that follows Moore’s law does help). In the next few years we’ll witness SSDs in read-intensive applications providing similar advantages to memory while accommodating much larger data sizes.

Syndicated via RSS From: http://www.asterdata.com/blog

Press the “pause” button for a minute. Everything from your TV to your Cell-Phone has gone interactive, companies today exist in a virtual world and they need your business every minute of the day. In this mad race to gain “customer” confidence, everyone is just about “social” i.e. having a facebook page, twitter accounts, “buzz” ing around you, connecting on orkut and eventually in “my(your)space”, interactive TV is just the beginning with 3D television. You are educated about anything by the company that wants your attention if not then your friend or an acquaintance who has more than 6 degrees of separation can influence your thoughts by being “linked-in” to your social network.

We need to measure all these events and calculate the “profitability” from being social (from an enterprise perspective). This is where Social Analytics becomes a “reality”. The ability of a company to measure how much “customer-centric” they are from being a “product” vendor and link that move to bottom line revenues is what we call Social Analytics.

It all started with the “long tail effect” and measuring its impact. Today we have cool tools that can accomplish the task, but these tools will form a solution only when implemented as a platform. Every vendor and solution integrator has claimed that they have a perfect solution and soon we will see a “magic” quadrant from Gartner on this subject and its vendor ratings et al.

In the next generation of the Data Warehouse, the measures, metrics and KPI’s for any company will be led by its “customer-centricity” which largely will be social analytics driven.  The new frontier of CRM+Marketing = Social Presence, will be accounted for in “Customer Revenue”+”Influencer Revenue” = Total Revenue, and Customer Profitability = Total Revenue / “Number of people influenced” (how many people did a customer influence to buy a service or product). Product and Service profitability will be driven by micro-segmented markets and neural networks will become a normal requirement to conduct business in that frontier.

The future is bright and the road to that future is being paved at a pace faster than your imagination.

Watch this blog for more information on this subject. 

Syndicated via RSS From: http://www.b-eye-network.in/blogs/krishnan/

• Lack of vision – The vision of the organization may be very narrow
• Lack of clarity – The organization is not clear about its customers needs and goals
• Lack of market – The organization is in a market where they cannot compete
• Lack of credentials – The organization may have a great strategy around services/solutions and no credentials to back up their solutions
• Lack of scalability – The organization may have limited scalability in terms of manpower, causing strategies to fail
• Lack of transparency – The organization may lack transparency in client communications with their teams and thereby causing failure
• Lack of capabilities – The organization may have limited capabilities in service offerings and hence may cause bandwidth constraints into itself.
• Lack of collaboration – The organization may not work with the client in a collaboration and may end-up performing as an order taker than a partner

In my perspective there are more reasons, but a combination of the reasons above is where normally the failure curve starts to creep up. I have been reading up on seminal work from great strategy gurus, who also recommend that if any of the above points are not clearly articulated, we will definitely see issues. My two cents, revisit the points and ponder over it to come to your conclusions.

Syndicated via RSS From: http://www.b-eye-network.in/blogs/krishnan/

In the Digital world, you have concerns about losing your identity, due to theft. But that applies only in certain categories of data, on the other hand the digital world also exposes your identity to a lot of people. You can change your name and fake every information about yourself, but the computers, the smartphones, the iPods etc have their own “identity” and you cannot change that particular identity, it is like the human “DNA” and will continue to provide clues in any situation – whether physical or cyber crime.

However the legalities in this case may turn out to be is not the focus here. In the futuristic world (5 years from now for discussion), the digital DNA will rule our lives more than ever. In that day and age, you can run but you cannot hide for long.

From a data practitioner perspective, this means more data to manage, and more applications to control. This is exciting and scary at the same time.

Syndicated via RSS From: http://www.b-eye-network.in/blogs/krishnan/

Virtualization is not cloud computing. However virtualization has significant potential when it is used to achieve cloud-like characteristics such as elasticity, economies of scale, accessibility, simplicity to deploy etc. I have always believed that the next wave of cloud computing is going to be all about solving “special” problems on the cloud – I call it a vertical cloud. These vertical problems could be in any domain, technology stack, or industry. Raw computing has come long way. It is about the right time we do something more interesting with the raw cloud computing.

Delphix is attempting to solve a specific problem – database virtualization. I met the CEO Kaycee Lai and the VP of sales Jedidiah Yueh at Under The Radar reception the night before. They have great background in understanding the cost and flexibility issues around de-duplication from their days at EMC. They have assembled a great team including Alok Srivastava from Oracle who ran Oracle RAC engineering prior to joining Delphix. Most large database deployments have multiple copies of single database that customers use for purposes beyond production such as staging, testing, and troubleshooting. This replication is expensive from process, resources, and storage perspective and takes long time to provision instances. The founders saw this problem first hand at EMC and decided to solve it.

At the core their offering is a read-write snapshot of a database. That’s quite an achievement. The snapshots are, well, snapshots. You can’t modify them. When you make this compromise they occupy way less space. Delphix took the same concept but created the writable snapshots and a seemingly easy to use application (I haven’t used it) that allows quick de-duplication based on these snapshots. You can also go back in time and start your instance from there.

Delphix has great value proposition in the database virtualization – help the customers reduce their hardware and people – DBA and system administrators – cost at the same time accelerate the IT processes. I like their conscious decision not to go after the backup market. Sometimes you have a great product but if it is marketed in the wrong category with vendors fighting in the red ocean you could die before you can grow. They had the best pitch at the conference – very calm, explaining the problem, articulating the value proposition, emphasizing right people on the team, and identifying the target market. If you are an entrepreneur (or even if you are not) check out their pitch and Q&A. There is a lot you can learn from them.

Syndicated via RSS From: http://cloudcomputing.blogspot.com/

Vendor hype cycles will continue to confuse you across the board. as the year grows, there will be more discussions on this subject by everybody. Your thoughts….

Syndicated via RSS From: http://www.b-eye-network.in/blogs/krishnan/

For a while I was attempting (probably the correct word) to keep both this blog as well as my SQL Server Magazine SQL Server BI blog current. Working on growing B.I. Voyage , speaking, writing, and blogging is quite the work load. And so this blog suffered as a result. THANK YOU all who read my blog here on SQLTeam.com, I intend to leave it live for those who might benefit from its content at a later date. My consolidated, single blog can be found at http://www.sqlmag.com/blogs/sqlserverbi.aspx .

Cheers,

Derek

Syndicated via RSS From: http://weblogs.sqlteam.com/derekc/Default.aspx