Oracle TimesTen
Analysis of in-memory DBMS TimesTen. Related subjects include:
- Oracle, which acquired TimesTen
- Memory-centric data management
Many kinds of memory-centric data management
I’m frequently asked to generalize in some way about in-memory or memory-centric data management. I can start:
- The desire for human real-time interactive response naturally leads to keeping data in RAM.
- Many databases will be ever cheaper to put into RAM over time, thanks to Moore’s Law. (Most) traditional databases will eventually wind up in RAM.
- However, there will be exceptions, mainly on the machine-generated side. Where data creation and RAM data storage are getting cheaper at similar rates … well, the overall cost of RAM storage may not significantly decline.
Getting more specific than that is hard, however, because:
- The possibilities for in-memory data storage are as numerous and varied as those for disk.
- The individual technologies and products for in-memory storage are much less mature than those for disk.
- Solid-state options such as flash just confuse things further.
Consider, for example, some of the in-memory data management ideas kicking around. Read more
Traditional databases will eventually wind up in RAM
In January, 2010, I posited that it might be helpful to view data as being divided into three categories:
- Human/Tabular data –i.e., human-generated data that fits well into relational tables or arrays.
- Human/Nontabular data — i.e., all other data generated by humans.
- Machine-Generated data.
I won’t now stand by every nuance in that post, which may differ slightly from those in my more recent posts about machine-generated data and poly-structured databases. But one general idea is hard to dispute:
Traditional database data — records of human transactional activity, referred to as “Human/Tabular data above” — will not grow as fast as Moore’s Law makes computer chips cheaper.
And that point has a straightforward corollary, namely:
It will become ever more affordable to put traditional database data entirely into RAM. Read more
Notes on the Oracle Database 11g Release 2 white paper
The Oracle Database 11g Release 2 white paper I cited a couple of weeks ago has evidently been edited, given that a phrase I quoted last month is no longer to be found. Anyhow, here are some quotes from and comments on what evidently is the latest version. Read more
Ordinary OLTP DBMS vs. memory-centric processing
A correspondent from China wrote in to ask about products that matched the following application scenario: Read more
| Categories: In-memory DBMS, McObject, Memory-centric data management, OLTP, Oracle TimesTen, solidDB | 7 Comments |
The other shoe finally drops for Oracle and BEA
As previously noted, I’ve been writing about an Oracle/BEA merger since 2002. So like many observers, I find I have little more to say on the subject. Let’s go straight to the bullet points: Read more
| Categories: HP and Neoview, IBM and DB2, Oracle, Oracle TimesTen, SAP AG | 2 Comments |
IBM acquires SolidDB to compete with Oracle TimesTen
IBM is acquiring Solid Information Technology, makers of solidDB. Some quick comments:
- solidDB is actually a very interesting hybrid disk/in-memory memory-centric database management system. However, the press release announcing the deal makes it sound as if solidDB is in-memory only.
- That strongly suggests that IBM is buying Solid mainly to compete with Oracle TimesTen. As of last June, solidDB was already IBM’s TimesTen answer via a partnership; this deal just solidifies that arrangement.
- This probably isn’t good news for Solid’s MySQL engine. That’s a pity, since solidDB technically has the potential to be the best MySQL engine around.
- Notwithstanding IBM’s presumed intentions, Solid’s main market success historically is as an embedded system in telecommunications equipment, network software, and similar systems.
- Last year I wrote a white paper on memory-centric data management, showcasing four products. IBM now has bought two of them, namely Solid’s and Applix’s (via Cognos).
- Comparisons to IBM’s embedded Java DBMS Cloudscape are pointless. That’s just a failed product vs. solidDB or Sybase SQL Anywhere, and IBM long ago cut its losses.
| Categories: Cache, Cognos, IBM and DB2, In-memory DBMS, Memory-centric data management, MySQL, OLTP, Oracle TimesTen, solidDB, Sybase | 5 Comments |
SolidDB caching for DB2
It’s just at the proof-of-concept stage, but Solid has a nice write-up about SolidDB being used as a front-end cache for DB2. Well, it’s a marketing document, so of course there’s a lot of pabulum too, but interspersed there’s some real meat as well. Highlights include 40X throughput improvement and 1 millisecond average response time (something that clearly can’t be achieved with disk-centric technology alone).
Analogies to Oracle/TimesTen are probably not coincidental; this is exactly the upside scenario for the TimesTen acquisition, as well as being TimesTen’s biggest growth area towards the end of its stint as an independent company.
| Categories: Cache, IBM and DB2, Memory-centric data management, OLTP, Oracle, Oracle TimesTen, solidDB | 1 Comment |
Oracle, Tangosol, objects, caching, and disruption
Oracle made a slick move in picking up Tangosol, a leader in object/data caching for all sorts of major OLTP apps. They do financial trading, telecom operations, big web sites (Fedex, Geico), and other good stuff. This is a reminder that the list of important memory-centric data handling technologies is getting fairly long, including:
- Object caching (e.g., Tangosol, Progress ObjectStore)
- In-memory RDBMS (e.g., Oracle TimesTen, Solid BoostEngine, McObject eXtremeDB)
- Stream processing (e.g., Progress Apama, Streambase)
And that’s just for OLTP; there’s a whole other set of memory-centric technologies for analytics as well.
When one connects the dots, I think three major points jump out:
- There’s a lot more to high-end OLTP than relational database management.
- Oracle is determined to be the leader in as many of those areas as possible.
- This all fits the market disruption narrative.
I write about Point #1 all the time. So this time around let me expand a little more on #2 and #3.
Read more
Is Oracle losing its edge?
Over in the Monash Report, I posed the question: Is Oracle losing its edge in DBMS? Here are some of the data points that make me suspect it has. A number of these points also apply to the other large mainstream DBMS vendors; a number, however, do not.
- Oracle lags behind both IBM and Microsoft on native XML. I can’t recall Oracle trailing both those vendors at once on anything as major before.
- I’ve come to believe that memory-centric technology is a really powerful improvement to DBMS. Oracle’s done nothing effective inhouse in this area. Buying TimesTen was a good move, however.
- I’m growing less inclined to excuse the ongoing complexity of Oracle database administration. From the classic Progress DBMS to Solid’s truly zero-DBA system (embed it in telecom equipment without a keyboard or monitor, and it runs for years) to ANTs’ new challenge, there are a whole lot of fairly full-featured RDBMS with a whole lot less knobs and dials. Do they match Oracle’s full functionality? No. But could they handle a lot of the apps which run on Oracle today? In some cases, quite possibly. So why hasn’t Oracle itself come up with a plug-compatible Oracle Lite that truly is lite?
- High-end data warehouse scaling continues to be a problem for Oracle. Yes, the effective upper bound on Oracle database size keeps increasing. But so does the size of the largest data warehouses, and the latter number is definitely greater than the former.
- Microsoft Analysis Services is quite cool. Why hasn’t Oracle done the same thing?
- There are many seemingly major feature advantages that Oracle has failed to exploit. Robust text processing in the DBMS and Virtual Private Database (a cool security feature) are two of the biggest that come to mind. They’ve sold well to customers who were predisposed to want them, but Oracle hasn’t had the stomach to broaden the market for them. Yes, that’s more of a marketing problem than a technical one, but after a decade of squandering your technical advantages, you can find that your people lose the enthusiasm to create more of them. What else has Oracle done in years that is as functionally innovative? Nothing comes to mind.
- The whole apps-vendor-consolidation strategy is often presented along with a huge white flag regarding DBMS market share growth.
- Oracle has never, ever solved its applications product problems. Ditto its BI development problems. Both these fiascos are plausibility arguments for the thesis that the company can NOT reverse bad product philosophy choices, no matter how much market evidence slaps them in the face, and no matter how adaptable the company is in other ways.
- Oracle’s analyst relations have gotten very defensive and insular. To some extent, that happens at most big companies. But I cannot recall a previous time when IBM’s and Microsoft’s DBMS operations were both more accessible than Oracle’s, and I’ve been a DBMS analyst since the industry’s very early days. Often when a company acts this bizarrely, it does so because it has a great deal to hide.
That’s a lot of evidence, even without mentioning threats from the open sourcers and the data warehouse appliance guys. So why am I not wholly convinced yet? Well, reasons include a variety of scalability features, extensibility features that are rivaled only by IBM’s, market share dominance on Linux, and Andy Mendelsohn. That’s a pretty compelling list too. Still, the Oracle colossus is teetering a little bit, and it’s not beyond imagination that some future earthquake could bring it crashing down.
| Categories: Oracle, Oracle TimesTen | 1 Comment |
Defining and surveying “Memory-centric data management”
I’m writing more and more about memory-centric data management technology these days, including in my latest Computerworld column. You may be wondering what that term refers to. Well, I’ve basically renamed what are commonly called “in-memory DBMS,” for what I think is a very good reason: Most of the products in the category aren’t true DBMS, aren’t wholly in-memory, or both! Indeed, if you catch me in a grouchy mood I might argue that “in-memory DBMS” is actually a contradiction in terms.
I’ll give a quick summary of the vendors and products I am focusing on in this newly-named category, and it should be clearer what I mean:
- TimesTen (now owned by Oracle): TimesTen is the quintessentional “in-memory DBMS.” It’s a fairly full relational DBMS, but if you want to persist memory to disk it has to be handed off to a conventional DBMS. Historically, that has usually been MySQL or Oracle. TimesTen’s biggest market penetration has been in financial trading.
- Solid Information Technology‘s BoostEngine: Solid is a Finnish company (or was — it’s pretty American now) specializing in embedded DBMS sold mainly for telecommunication uses. Big OEM customers include several well-known telecom equipment manufacturers and HP (for OpenView). “Embedded” often means no DBA, no monitor, no keyboard — they box manufacturer installs it and there it stays for the life of the product. Solid has to offer strong replication capabilities, since its products are often used in highly distributed (e.g., multiblade, multibox) environments. So it’s taken the next step and exploited the replication by allowing customers to use some instances of the product disklessly.
- Event-stream products from Streambase and Progress: The canonical application for event-stream products is automating financial trading decisions based on the flow of market information. Mike Stonebraker, the brains behind Streambase, has recently popularized the idea; Progress bought Apama, who actually have been in the business longer. These applications require even more speed than the financial trading apps that TimesTen handles, and they discard most of the information they look at. In-memory is the only way to go.
- Progress’s ObjectStore: ObjectStore comes from the company Object Design, which merged into Excelon, which was acquired by Progress. It’s really a toolkit for building DBMS and similar systems, which is why it’s at various times been marketed as an OODBMS and an XML DBMS, without a lot of success either way. But there have been a few sterling apps built in ObjectStore even so, including a key part of the Amazon bookstore Despite this limited market success, a significant fraction of Progress’s best engineering talent has moved over to the Real-Time Division to focus on ObjectStore and other memory-centric products. The memory-centric aspect of ObjectStore is this: ObjectStore’s big virtue is that it gets objects from disk to memory and vice-versa very efficiently, then distributes and caches them around a network as needed. This was originally invented for client/server processing, but works fine in a multi-server thin client setup as well. And object processing, of course, relies on a whole lot of pointers. And pointer-chasing is pretty much the worst way to deal with the disk speed barrier, unless you do it in main memory.
- Applix‘s TM1: Like many companies in the analytics area, Applix has had trouble deciding whether it sells applications, BI system software, or both. But in any case its core technology is TM1, a memory-centric MOLAP offering. Traditional MOLAP products reside on the horns of a nasty dilemma: They rely on precalculation to give good performance, but that causes ghastly database explosion. Applix gets out of this problem by doing no precalculation whatsoever, loading the data into main memory, and executing all queries on the fly.
- SAP’s BI Accelerator: SAP is building out an elaborate technology stack with NetWeaver, especially in the BI area. One important aspect is that the full data warehouse is logically broken (or copied) into a series of data marts called “InfoCubes.” BI Accelerator takes the logical next step, loading an entire InfoCube into main memory. Almost every query is executed via a full table scan, which would be insane on disk but makes perfect sense when the data is already in RAM.
So there you have it. There are a whole lot of technologies out there that manage data in RAM, in ways that would make little or no sense if disks were more intimately involved. Conventional DBMS also try to exploit RAM and limit disk access, via caching; but generally the data access methods they use in RAM are pretty similar to those they use when going out to disk. So memory-centric systems can have a major advantage.