Please note: This blog post is for our French-speaking readers only.

Julien Pauli has just published a great commentary about Zend_Service_LiveDocx, comparing it to his library odtphp. If you are able to read French take a look at the following:

Voila quelques temps j’ai sorti odtphp, un script PHP qui analyse des fichiers odt à la recherche de “tags” et qui les remplace par des données que vous fournissez.
C’est très pratique pour générer des factures ou d’autres documents basés sur un template odt et une source de données type MySQL.

Continue reading:

• LiveDocx : service web de templating de documents

Please note: This blog post is for our German-speaking readers only.

Gjero Krsteski has just published an introduction that illustrates how to create word processing documents using Zend_Service_LiveDocx. If you are new to Zend_Service_LiveDocx and can read German, this is a great starting point:

Letztens stand ich vor der Herausforderung, eine PDF-Klasse zu schreiben, die PDF-Dateien erzeugt. Als Inhalt für die PDF Dateien waren sensible Kundendaten angedacht. Diese Anforderung war mir nicht fremd, jedoch konnte ich mich gut daran erinnern, dass sie mit viel Arbeit verbunden war. Die meisten Libraries bieten nur die Möglichkeit, die PDF-Dateien Zeile pro Zeile zu generieren Je nachdem woher man die Daten dafür beschaffen muss, kann sich dies zu einem mühseligem Unterfangen entwickeln. Hier kann ich aktuell zwei Lösungsansätze vorschlagen, die leicht anzuwenden sind und eine gute Sicherheit gewährleisten.

Continue reading:

• PDF Datei erzeugen

Faheem Abbas has just published a great little tutorial that illustrates how to create word processing documents using Zend_Service_LiveDocx. If you are new to Zend_Service_LiveDocx, this is a great starting point:

You may have heard about FPDF, TCPDF. These are used to generate PDF document in PHP. But very few of you may have heard about LiveDocx, can be found here http://www.livedocx.com/. The most cool news I’m going to tell you that Zend Framework 1.10.0 provides clean and simple interface to LiveDocx API. LiveDocx is soap based service. If you are interested more in LiveDocx, following the link I’ve already provided you.

Continue reading:

• Creating word processing document using Zend_Service_LiveDocx

I am delighted to announce that the final, stable version of Zend_Service_LiveDocx in Zend Framework 1.10 has just been released.

Download Zend Framework 1.10.0 Full and read the updated documentation on ZendFramework.com.

You can find Zend_Service_LiveDocx_* at the following locations in the distribution tree:

Source code

/library/Zend/Service/LiveDocx.php
/library/Zend/Service/LiveDocx/MailMerge.php
/library/Zend/Service/LiveDocx/Exception.php

Unit tests

/tests/Zend/Service/LiveDocx/LiveDocxTest.php
/tests/Zend/Service/LiveDocx/MailMergeTest.php
/tests/Zend/Service/LiveDocx/MailMerge/*

Demonstration applications

/demos/Zend/Service/LiveDocx/*

I really recommend taking a look at the demonstration applications, as they illustrate all of Zend_Service_LiveDocx functionality.

You can check that your server environment is set up correctly to run Zend_Service_LiveDocx, by executing check-environment.php in the directory /demos/Zend/Service/LiveDocx/.

I would like to thank Matthew Weier O’Phinney, Zend Framework Project Lead and all other members of the Zend Framework community, who have contributed to Zend_Service_LiveDocx.

The first implementation of LiveDocx in PHP was in a family of classes called Tis_Service_LiveDocx. It was necessary to give the classes the prefix Tis_ as the Zend Framework coding standards allow the prefix Zend_ to be used only for classes, which are part of the official Zend Framework.

As of the first alpha version of Zend Framework 1.10, the functionality of Tis_Service_LiveDocx is available directly in the Zend Framework in Zend_Service_LiveDocx.

As of today, the use of Tis_Service_LiveDocx is deprecated. Instead use Zend_Service_LiveDocx.

With the exception of the constructor, the API of Zend_Service_LiveDocx is 100% backward compatible to that of Tis_Service_LiveDocx.

At the wish of the Zend Framework community, the constructor was updated from:

1
2

// deprecated - do not use
$phpLiveDocx = new Tis_Service_LiveDocx_MailMerge('myUsername', 'myPassword');

to:

1
2
3
4
5
6
7
8
9
10
11
12
13

// correct - do use
$phpLiveDocx = new Zend_Service_LiveDocx_MailMerge(
    array (
        'username' => 'myUsername',
        'password' => 'myPassword'
    )
);
 
// alternatively
$phpLiveDocx = new Zend_Service_LiveDocx_MailMerge();
 
$phpLiveDocx->setUsername('myUsername')
            ->setPassword('myPassword');

I would encourage you to update your projects to use the Zend_Service_LiveDocx family of classes.

If you need any assistance in migrating your code, please post a support request into the support forum.

You can now download the first beta version of the Zend Framework that contains Zend_Service_LiveDocx. The final stable version will be released in the next few days.

Please go to the download section of ZendFramework.com and download Zend Framework 1.10.0 Beta 1 Full.

The paths to the demonstration applications and documentation remain the same as listed in the previous post.

The backend server guys have just published an official maintenance schedule in their blog:

• Patch Monday: LiveDocx Server Update Policy

In these maintenance windows, the backend service will not be available. Usually, the downtime is only a few minutes.

You can now download the first alpha version of the Zend Framework that contains Zend_Service_LiveDocx. The final stable version is marked for release on January 26, 2010.

Please go to the download section of ZendFramework.com and download Zend Framework 1.10.0 Alpha Full.

The paths to the demonstration applications and documentation remain the same as listed in the previous post.

In the next step of phpLiveDocx becoming a part of the official Zend Framework distribution file, I am delighted to announce that the code has now arrived in the Zend Framework Standard Trunk repository. This is the last step before it becomes part of the distribution file.

Using Subversion (SVN) you can anonymously checkout the Standard Trunk repository:

svn checkout http://framework.zend.com/svn/framework/standard/trunk/ ./

You can find Zend_Service_LiveDocx_* at the following locations:

Source code

/library/Zend/Service/LiveDocx.php
/library/Zend/Service/LiveDocx/MailMerge.php
/library/Zend/Service/LiveDocx/Exception.php

Unit tests

/tests/Zend/Service/LiveDocx/LiveDocxTest.php
/tests/Zend/Service/LiveDocx/MailMergeTest.php
/tests/Zend/Service/LiveDocx/MailMerge/*

End-User documentation (as raw XML)

/documentation/manual/en/module_specs/Zend_Service_LiveDocx.xml
/documentation/manual/en/figures/zend.service.livedocx.*

Demonstration applications

/demos/Zend/Service/LiveDocx/*

Learn more about the Zend Framework repository in the official wiki.

I would like to thank Matthew Weier O’Phinney, Zend Framework Project Lead for promoting Zend_Service_LiveDocx to the Standard Trunk repository.

The next and final step will be Zend_Service_LiveDocx becoming part of the Zend Framework 1.10 distribution file.

I just received an e-mail newsletter from the backend LiveDocx guys, announcing the immediate availability of their new Hosted ‘Shared’ Service.

This new service offers all the functionality of the free public LiveDocx service, but in addition adds a number of enterprise features:

• Only 16 users per server.
• Dual-Core AMD Opteron Processor 1218 HE 2.60 GHz.
• 64 bit and 4 GB RAM.
• High availability (99.9% uptime).
• Blisteringly fast Internet connection (130 Gbit).
• Faster round trips.
• Privacy regulations complaint.

The Hosted ‘Shared’ Service is available to rent for only EUR 29.90 (approx. USD 44.47) per month.

For details, take a look at the new pricing section on LiveDocx.com.

The ingredients for life as we know it have been found in the Orion Nebula.

By finely separating the spectrum of incoming light, astronomers are able to detect the chemical fingerprints of molecules like water and methanol. The spectrograph that their work produces can be seen in the image above. The peaks represent the presence of the molecule indicated.

The new data was collected by the Herschel Telescope, launched into space last year by the European Space Agency. Herschel’s HiFi instrument uses a new technique to do more-sensitive spectroscopy. It will enable scientists to better understand the chemistry of space.

The Orion Nebula is located about 1,300 light-years away. No very active star-forming region is closer to Earth. M42, as the nebula is also known, is 24 light-years across.

Wired (Thanks DG)

A 101-year-old woman in China has baffled doctors after growing a huge goat-like horn on her forehead.

Zhang Ruifang claims the growth first appeared only last year and has since expanded to be more than 6cm long. Now her family in Linlou, Henan province, are concerned about a second mark on the other side of her forehead.

“[At first] we didn’t pay too much attention to it,” said Mrs Zhang’s youngest son Zhang Guozheng. ”Now something is also growing on the right side of her forehead — it’s quite possible that it’s another horn.”

9 News

Street signs warning Romanian drivers to be careful of drunken pedestrians lying on roads were erected by road safety chiefs worried about the “despairing” levels of accidents.

Officials in Pecica, a village town about 13 miles from the Hungarian border in the country’s west, ordered the bright red signs, complete with the phrase “Attention – Drunks”. The 10 road signs, which also show a person crawling on their knees while clutching a glass in one hand, were erected in popular nightspot areas close to the city’s bars and restaurants.

Telegraph

“It may seem odd, but the locals swear it’s true. People in a Japanese mountain region have reported a number of kangaroo sightings, and journalists are now trying to stalk the marsupials.

The descriptions given by the apparent eyewitnesses seem close enough. For years they have spoken of a beige animal with large ears, one to 1.5 metres (three to five feet) tall, that stands by the roadside and then hops away.

The sightings were all reported in the Mayama mountain district of Osaki city in Miyagi prefecture, a community of 441 households, located about 350 kilometers (220 miles) north of Tokyo.

The city has received about 30 reports of ‘kangaroo-like animals,’ including three cases since December, when the mountain area was often covered in snow, said local official Tetsuya Sasaki.”

Read more at Yahoo

“Simple motor actions, like moving marbles upward or downward between two cardboard boxes, may not seem meaningful. But a study published April 2010 in Cognition shows that motor actions can partly determine people’s emotional memories.

Moving marbles upward caused participants to remember more positive life experiences, and moving them downward to remember more negative experiences, according to Daniel Casasanto (MPI and Donders Institute, Nijmegen) and Katinka Dijkstra (Erasmus University, Rotterdam). ‘Meaningless’ motor actions can make people remember the good times or the bad.

When people talk about positive and negative emotions they often use spatial metaphors. A happy person is on top of the world, but a sad person is down in the dumps. Some researchers believe these metaphors are a clue to the way people understand emotions: not only do we use spatial words to talk about emotional states, we also use spatial concepts to think about them.”

Read more at Science Daily

“The Church of Scientology is expected to come under closer scrutiny over the coming days.

The Senate is preparing to vote on a possible inquiry into alleged abuses against Australians which have occurred within the organisation.

Tonight’s Four Corners program details claims of mistreatment and allegations that some women were pressured to have abortions.

Former members in Australia and in the United States have spoken openly for the first time about their lives in the Church.

Scientology has denied their claims but their stories raise more questions about whether the Church of Scientology should keep its status as a tax free charity.

Emily Bourke compiled this report.

EMILY BOURKE: The former members of the Church of Scientology who’ve spoken out were members of an elite religious unit known as the Sea Organisation.

One former Sea Org member whose identity will be revealed in tonight’s Four Corners program has detailed allegations of a strict regime of discipline and punishment in place during the 1960s.

SEA ORG MEMBER: I mean looking back I, you know, I deeply regret my, even my fringe participation in some of the things that went on. And I’m ashamed of some of them.

People were thrown overboard. Hands bound and feet bound and blindfolded. You know women of 55-years-old, you know, for, for, for running a process incorrectly. A counselling technique incorrectly in a, in a auditing session, you know.”

Read more at ABC News (thanks, Fosca)

I arrived at Bristol to find a note in my dressing room from Dara O’Briain wishing me enjoyable shows with the bright and energetic crowds of Bristol. And he was very right in his description. Bristol is famously a great house to play: the roar when I came on stage was long and deafening, and audience and participants alike were fantastic. The first night it really took me by surprise and I hugely enjoyed myself. The second night, the adrenalin wasn’t there so much and I think I was a little under par, and then the third was good fun again.
We stayed in the wonderful Hotel Du Vin, which kicks the ass of any other hotel on tour. Impeccable.
Friday we went to the Zoo and had a great tour day out. Saturday was tea round at Peter Clifford’s, whom some of you will know from The Devil’s Picturebook and The Heist. Others of you may know my dear friend from his roles in the stunningly good Shakespeare at the Tobacco Factory seasons. They’re about to do The Tempest and Midsummer Night’s Dream in the gorgeous Tobacco Factory theatre where I got started, so do go along if you can. It was a wonderful stay in my beautiful University home.

We are now in Eastbourne. It’s a very different crowd, but the shows have been good so far. A good friend has come over from the States to see the show (and Andy’s Ghost Stories) and today we had a bloating pub lunch in the nearby village of Alfriston, which I may have spelt correctly. Our hotel is a stranger to wi-fi, so I have been slow on blog entries. I type this, as I tend to tweet, face down in a steamer sat in my dressing room.

Excitingly, I am trying out something new in the show. It’s a new ending to one of the pieces that felt like it needed it. It’s really enjoyable to let it settle in and make these sorts of changes. Keeps one on ones tootsies.

ta-ta
x

“Almost 60 acres. 530 x 450 meters in size. Created in an evening of August, 2009, at the province of Zeeland, Netherlands, it’s the biggest “crop circle” ever created. And with a twist.

Everything was recorded in video, as this was definitely a very human creation. To be more exact, a creation of 60 humans captured in its process from concept to realization by the cameras of the reality TV show “Try Before You Die”.

The culprits are the members of XL D-Sign team, which has been creating fantastic formations for more than ten years – many of which are promoted as “mysterious” to this day. This latest one, the biggest one to date, was properly named project Atlas, and aimed not only to break the size record but also depict “a message of both the beauty and vulnerability of man”.

The gigantic formation can be interpreted in several ways, from the metamorphosis of a butterfly, to the Vitruvian Man, to Mothman and perhaps even chakras. All part of a human symbology, with a human message, created by humans to humans, surpassing in size every crop circle ever created.”

Read more at Forgetomori (thanks, DG)

“‘Il Cavallo,’ the huge equine statue Leonardo Da Vinci never got to make, wasn’t plagued by technical problems as was widely believed, a new multidisciplinary research has revealed.

On the contrary, Da Vinci’s plan for the largest equestrian statue in the world was a perfectly feasible project which, if completed, would have probably been his greatest legacy, more than ”The Last Supper” or any other work.

Commissioned in 1482 by Lodovico Sforza, duke of Milan, in honor of his father Francesco, the massive bronze horse took Leonardo 17 years of research, but was never completed.

Indeed, when the full-scale clay model was finally ready to be cast in a single operation in 1499, all the needed bronze was used to make cannons for an imminent war against the King of France.

The molds were lost and the clay model was reduced to rubble by the invading French soldiers.

Although Leonardo never stopped mourning the ‘horse-that-never-was,’ engineers have always believed the daring plan to make the largest single-pouring cast ever would have failed because of technical problems.”

Read more at Discovery News (thanks, ReliegiousMarie)

“There is a glacier in Antarctica that seems to be weeping a river of blood. It’s one of the continent’s strangest features, and it’s located in one of the continent’s strangest places — the McMurdo Dry Valleys, a huge, ice-free zone and one of the world’s harshest deserts.

A bleeding glacier. Discovered in 1911 by a member of Robert Scott’s ill-fated expedition team, its rusty color was at first theorized to be caused by some sort of algae growth. Later, however, it was proven to be due to iron oxidation. Every so often, the glacier spews forth a clear, iron-rich liquid that quickly oxidizes and turns a deep shade of red. According to Discover Magazine –

The source of that water is an intensely salty lake trapped beneath 1,300 feet of ice, and a new study has now found that microbes have carved out a niche for themselves in that inhospitable environment, living on sulfur and iron compounds. The bacteria colony has been isolated there for about 1.5 million years, researchers say, ever since the glacier rolled over the lake and created a cold, dark, oxygen-poor ecosystem.

Even weirder: scientists think that the bacteria responsible for Blood Falls might be an Earth-bound approximation of the kind of alien life that might exist elsewhere in the solar system, like beneath the polar ice caps of Mars and Europa.”

Read more at Mental Floss

Rosalie Marshall, vnunet.com, Wednesday 28 January 2009 at 18:02:00

QlikView 9 planned for May or June 2009

Independent business intelligence firm QlikTech has added new support to its QlikView product for HP Neoview, and has given customers a preview of its product roadmap for 2009.

The new support will allow customers with large data volumes in an HP data warehouse to quickly visualise the data and make faster decisions, according to the QlikTech. The vendor had previously announced support for warehouse applications supplied by Sybase IQ and Kalido.

"We are working with a number of other next-generation data warehouse providers, for which we may announce support in the future," said Anthony Deighton, senior vice president of products at QlickTech.

The firm has also been expanding its partnerships with front-end application suppliers in OEM agreements, including with Atex, to allow businesses to view data in a way best suited to their needs.

QlikView 9, due out in May or June this year, will take the product to even more places, according to Deighton. "A strategy called 'QlickView everywhere' will make it easier to do mash-ups and portals," he added.

Additionally QlikView 9 will make it easier for customers to make larger deployments of the software, and offer usability enhancements such as new ways to visualise and simplify data.

"We intend to build in visual analysis and make the output interactive, showing data in a way that makes it easy to answer particular questions," Deighton said.

"We always think of QlikView as a simple product that normal human beings can use, but we never stop thinking about how to make it even simpler and easier to use."

Rosalie Marshall, IT Week, Friday 8 August 2008 at 16:25:00

Fujitsu Services survey finds lack of advice on what CIOs should be included in carbon footprint

Chief information officers (CIO’s) are split on whether outsourced IT operations should count towards an organisation’s total carbon footprint, according to a Fujitsu Services survey.

While the majority of the CIOs preferred to count outsourced IT operations as part of their carbon footprint, almost a quarter believed the opposite and wanted the contracting company to become responsible. The survey was of 100 CIOs in UK companies employing more than 1000 people.

Fujitsu Services, as a provider of outsourcing services, said industry leaders should work together to agree common standards for calculating carbon footprints.

“This research suggests that many are erring on the side of caution, preferring to double-count rather than risk understating environmental impact,” the Fujitsu Services report noted.

Dave Bailey, IT Week, Friday 8 August 2008 at 12:43:00

Cable provider says it could deliver 200Mbit/s services within four years.

Virgin Media is touting the possibility of providing customers with broadband speeds of up to 200Mbits/s within the next four years.

Speaking at a conference call on Virgin Media's second quarter earnings, chief executive Neil Berkett said, "Even by 2012, we'll still have an advantage to BT's fibre to the cabinet (FTTC). We'll certainly be able to offer up to 200Mbit/s broadband download speeds, if we so chose."

Also at the presentation Virgin Media unveiled its intention to launch a mobile broadband service in the fourth quarter of 2008, but with no further details. The service would launch on the back of re-negotiated wholesale rates with its mobile partner T-Mobile.

T-Mobile upgraded its 3G network from high speed downlink packet access (HSDPA) to high speed uplink packet access (HSUPA) in July. Although limited to users within the M25, uplink and downlink speeds would effectively be around 1Mbit/s and 2Mbit/s respectively.

Elsewhere, reports emerged that Virgin Media may be intending to withdraw from the corporate communications market to concentrate on its cable TV and broadband business. Investment bank Goldman Sachs is reported to have a potential merger plan for the business part of Virgin Media, NTL Telewest Business.

But a Virgin Media spokesman refused to be drawn on the reports. "We never comment on rumour or speculation," he said.

Rosalie Marshall, IT Week, Friday 8 August 2008 at 11:44:00

Google announces in quarterly report its AOL investment could be impaired

Google looks likely to take a shower on its $1bn investment in AOL, as documents filed with the US Securities and Exchange Committee say that investment is "impaired".

Google paid $1bn for a 5 per cent stake in AOL in 2005 but now concedes that its value has dropped substantially. But Google is still optimistic that the wilting web giant may recover. "We do not believe that such impairment is other than temporary,” noted Google in the report.

Nevertheless, Google may well have to write down that investment in the near future. . “There can be no assurance that impairment charges will not be required in the future,” it said.

Originally, Google invested in AOL to guard against the possibility of loosing one of its largest advertising partners to Microsoft.

“We expect that Microsoft will increasingly use its financial and engineering resources to compete with us,” Google noted in its 10-Q filing. “[Microsoft] can use [its] experience and resources against us in a variety of competitive ways, including by making acquisitions, investing more aggressively in research and development and competing more aggressively for advertisers and web sites.”

Dave Bailey, IT Week, Thursday 7 August 2008 at 17:49:00

Government to invest £5m into identity systems research

The UK government is to invest £5.5 million in developing the next generation of secure identity management systems.

The investment, led by the Technology Strategy Board, Engineering and Physical Sciences Research Council (EPSRC) and Economic and Social Research Council (ESRC), will create three new research projects, called EnCoRe, VOME and Privacy Value Networks.

The projects will be collaborations between business, academia and the public sector and will aim to ensure that the next generation of identity management systems strike appropriate balances with privacy concerns, the Technology Strategy Board said.

“In order to prepare UK businesses for competition in this global market, practical and cost effective solutions need to be developed which inspire public confidence by improving privacy and enabling consent as an integral part of future procurements," said Iain Gray, chief executive of the Technology Strategy Board.

Project EnCoRe will focus on providing more rigorous means for individuals to grant and revoke consent for the use, storage and sharing of personal data.

VOME will give a clearer hardware and software requirement for end users’ ideas and concepts regarding privacy and consent.

The final project, Privacy Value Networks (pvnets), aims to generate, a detailed understanding of individuals’ and organisations’ conceptions of privacy and identity.

EPSRC chief executive Professor David Delpy said that the new research had a unique approach, "looking at both the technological advances that need to be made alongside the social considerations and implications. The long term aim is to ensure a good balance between freedom and security for everyone.”

Rosalie Marshall, IT Week, Thursday 7 August 2008 at 17:31:00

Gartner recommends IT invest in information-access technologies

IT chiefs have been urged to go on a storage diet. Analyst firm Gartner recommends that enterprises are better served by the introduction of so-called information-access technologies, rather than the current penchant to splurge on cheap storage.

The fear of litigation combined with falling storage costs has encouraged many chief information officers to store masses of data, resulting in unnecessary business expense, argued Whit Andrew, analyst, Gartner in a new report.

The introduction of information-access technology, such as enterprise search, content classification and categorisation, and clustering would be far more efficacious, he said.

“With information access technology, companies that previously made retention decisions based on intuitive judgements about what was important can now designate critically on more advance approaches for measuring the value of content,” said Andrews.

Information-access technology will pay for itself when organisations do not have to upgrade their storage requirements so quickly, he added.

Vendors are also offering data storage services to help companies sort out what data they need to store, said Steve Murphy, managing director Hitatchi Data systems, a firm that offers such a service.

Hitatchi has begun offering customers two years of free storage upgrades, including the use of technologies such as virtualisation and de-duplication, to ensure companies make better use of their storage infrastructure, Murphy added.

Rosalie Marshall, IT Week, Thursday 7 August 2008 at 16:07:00

IT will be able to reduce its compliance costs

Central government is to make individual departments accountable for the business cost of introducing new legislation, under new proposals issued by the Department for Enterprise and Regulatory Reform (BERR). Those proposals also accentuate the pivotal role IT will play in ensuring that compliance is affordable.

BERR intends to compel government departments to make business-cost assessments of any new legislation. Departments will be allocated annual budgets to ensure those costs do not spiral out of control.

"If the UK is to remain a respected place to do business, we must not expect business simply to absorb the costs of a stream of new Government initiatives,” said John Hutton, secretary of state for business.

"This innovative approach will build on efforts already in place to cut the burden to business by 25 per cent by 2010."

Under the proposals, government departments will publish best-practices guides to ensure the task of automating compliance procedures is as effortless as possible.

Consutlation on the proposed budget system is now underway, and the first budgets could be set later this year.

Gareth Morgan, IT Week, Thursday 7 August 2008 at 12:55:00

Next-generation database software released to manufacturing

Microsoft's next-generation database software SQL Server 2008 has been released to manufacturers.

The upgrade will provide enhanced data handling capabilities, Microsoft claimed, adding enterprise-class business intelligence and data warehousing tools out of the box.

The ability to scale is one of the key new features of SQL Server 2008, said Ted Kummert, Microsoft's corporate vice president of data and storage platforms. SQL Server 2008 is far more scalable, improves performance and can handle higher levels of user concurrency, he added.

The release is also a significant milestone for Microsoft in terms of its overall software licensing strategy.

Microsoft made predictable and regular product upgrades a cornerstone of its Software Assurance programme, and the firm has fulfilled its obligations by delivering SQL Server 2008 within the anticipated 24-36 month window.

"Wow, this is a great day," said Dan Jones, SQL Server group programme manager at Microsoft, "we nailed it."

SQL Server 2008 is available immediately to download to MSDN and TechNet subscribers.

It will be offered in several editions, such as Enterprise, Workgroup and Standard. And for the first time, Microsoft is offering a version specifically for the web, which will be tuned for partners wanting to offer a hosted version of SQL Server 2008.

Other new features include: improved ability to load data from Oracle, Teradata and SAP NetWeaver; enhanced security features, allowing adminstrators to encrypt data and provide audit trails, policy-based managed, enabling users to enforce group policies; and support for spatial data, for compatibility with location-based applications.

Dave Bailey, IT Week, Wednesday 6 August 2008 at 16:26:00

Contract negotiations underway, but no break-up plans yet, says computer maker

Fujitsu Siemens Computers has moved to quash reports of its impending break up, describing the talk as "baseless".

Earlier, the Wall Street Journal had reported that leaders at Siemens were looking to dissolve the partnership, citing unnamed sources.

Fujisu Siemens' head of corporate communications dismissed that suggestion but admitted that all parties were in contract discussions.

The original partnership agreement started in 1999, he explained, and was due to run for 10 years. "If nobody sells its stake, then the partnership will continue for a further five years," he added.

Should Siemens not renew that partnership agreement, Fujitsu has first refusal over its 50 per cent stake.

And as the German engineering giant looks to restructure, as it aims to reduce operating costs by €1.2b by 2010, several of its partnership agreements are under the spotlight.

Dave Bailey, IT Week, Wednesday 6 August 2008 at 14:41:00

New service to help IT manage mobile devices

Mobile operator Orange has launched its hosted Device Management service which allows IT teams to manage and support devices mobile remotely.

The system enables IT to update, troubleshoot and even lock or wipe devices remotely and requires no back-end servers.

Security features included in the system allow IT administrators to remotely enforce password policies, lock down Bluetooth and camera functionality, and remove unauthorised applications, which could be affecting the mobile device's performance.

Managers can also apply company-specific settings over the air to all devices simultaneously, and can view logs of all user activity.

"It is becoming increasingly important for organisations to be able to manage their device fleet in a cost-effective and time-efficient way," said Anthony Keyworth, director of business products,at Orange's UK business services division.

Orange Device Management costs £3.50 + VAT per user, per month plus data charges, including Orange technical support.

Phil Muncaster, IT Week, Wednesday 6 August 2008 at 10:55:00

BSI and The Content Group launch new Steering Group

A new steering group has been launched with the aim of creating the first ever publically available specification (PAS) for enterprise content management applications, to create more clarity for IT buyers around the term.

The new Steering Group will be led by ECM consultancy The Content Group and the BSI British Standards to create a PAS by the end of the year, according to The Content Group. The PAS is a vital first step on the road towards an internationally agreed standard, accordint to Content Group founder, Ben Richmond.

Members of the steering group have been gathered from industry associations, analyst firms such as CMS Watch, system integrators like Logica, technology vendors and end user organisations including Credit Suisse.

"Unless organisations take a step back from the technology and adopt a best practice approach to defining and implementing an ECM strategy, the potential of this powerful solution set will never be realised," said Richmond in a statement. "This standard will provide a clear frame of reference for those engaging in ECM initiatives.”

Rosalie Marshall, IT Week, Wednesday 6 August 2008 at 10:46:00

Microsoft launches Active Protection Programme and an Explotability Index

Microsoft has confirmed it will give security vendors advance notice of vulnerabilities that it intends to address in its monthly patches, in order to provide users with better safeguards.

Microsoft's Active Protection programme will reduce the chances of cyber criminals outpacing the security professionals, said George Stathakopoulos, Microsoft general manager of security engineering and communications,

It will also issue a new Exploitability Index, which will provide customers with early information on the likelihood of exploit code being developed.

Previously security professionals had to wait for Microsoft’s monthly security update process to address vulnerabilities.

“As security threats become more sophisticated, the global security community must combine its resources and work together to provide maximum security protections to worldwide internet users,” said Stathakopoulos in a statement.

Phil Muncaster, IT Week, Tuesday 5 August 2008 at 15:38:00

New technologies will help improve user productivity, says Symantec

Security and systems management giant Symantec has beefed up its virtualisation offerings with new technologies it plans to acquire with the purchase of virtual workspace management firm nSuite.

The proposed acquisition, slated to be completed by the end of the month, will give Symantec presentation virtualisation technology, which enables staff to use applications running on remote servers as if they were running locally.

Symantec will also gain nSuite's expertise in connection broker technology, which allocates resources such as user profiles and applications to the endpoint, whether virtual or traditional.

NSuite's heritage has been in selling its products to hospitals. There, medical staff are able to take advantage of the technology which can virtualise their data and applications, port them to which ever desktop they happen to be working on at the time, and remove them from the desktop once they move on.

“The real promise of endpoint virtualisation is improving the end-user experience while helping to lower the cost of managing endpoint devices,” said Ken Berryman, vice president of endpoint virtualisation at Symantec. “Symantec’s strategy is to help enable a truly dynamic endpoint where applications and information are delivered to any computing environment in a seamless manner."

Dave Bailey, IT Week, Tuesday 5 August 2008 at 15:16:00

Help desk tools to be integrated with other on-demand apps

Secure remote access vendor LogMeIn today published the application programming interface (API) for its on-demand support tool, which it claimed will make it easier for customers to integrate the tool with third-party applications.

Firms will be able to better serve the needs of remote workers by integrating the LogMeIn Rescue tools with other critical business applications, such as customer relationship management systems, said Kevin Bardos, vice president for product management at LogMeIn.

"The API makes it easy to connect Rescue with other critical on-demand tools, " he said.

LogMeIn Rescue was first made available on the Force.com AppExchange from Salesforce.com in July. LogMeIn's Rescue API available now to current LogMeIn Rescue subscribers at no extra cost.

Dave Bailey, IT Week, Tuesday 5 August 2008 at 14:20:00

Open source networking takes a step forward with enterprise-class appliance

Open source vendor Vyatta has begun shipping a mid-range firewall and routing appliance, the Vyatta 2501, which it claims offers unparalleled price-performance.

The Vyatta 2501 is intended to meet the connectivity and security demands of medium and large enterprises, but costs a fraction of proprietary alternatives.

"The Vyatta 2501 combines features, performance, flexibility, and value on high-performance hardware, making it a clear alternative in a market saturated with expensive proprietary solutions," said Kelly Herrell, chief executive, Vyatta in a statement.

The 2501 uses Vyatta's open-network operating system and has two integrated gigabit Ethernet ports for LAN connections, together with two expansion slots, one PCI-X and one PCIe, for additional LAN/WAN connectivity.

The 1U high 2501 appliance has support for internet routing protocols, the Border Gateway Protocol (BGP), Open Shortest Path First (OSPF) and the Routing Information Protocol (RIP).

The appliance has gigabit LAN connectivity, with an option for 10 gigabit Ethernet optical connections. The wide area network interfaces range from ADSL, up to T1/E1 and T3 connections, and can be load-balanced.

The 2501 also incorporates a firewall, IPSec and point-to-point tunneling (PPTP) virtual private network (VPN) connections, and has integrated the Wireshark network packet analysis and capture system.

Vyatta say that the 2501 is available now, starting at £1,200 ($2,347) including service and support.

Dave Bailey, IT Week, Tuesday 5 August 2008 at 13:58:00

Orange's Network Boost guarantees application performance across the globe

Mobile operator Orange has launched Network Boost which promises to make it easier for customers to manage enterprise applications running over its wired network.

Network Boost can be used by firms to prioritise, accelerate and optimise specific applications, with Orange guaranteeing uptime. This will enable customers to deploy applications across the globe, while managing them through a single service desk.

Orange also claimed that Network Boost can improve incident management, and will provide guaranteed time-to-repair service level agreements.

Other features include the ability to compress data to minimise network delays, as well as monitor and anticipate network performance degradation.

Orange said that the new service would be available in 220 countries.

Gareth Morgan, IT Week, Tuesday 5 August 2008 at 13:29:00

Sybase touts appliance as easy way to tackle "exploding" data volumes

Database vendor Sybase has today released a new analytics appliance, which it claims will help users rapidly glean business-enhancing intelligence from over-burdened data warehouses.

The Sybase Analytic Appliance combines the vendors IQ data warehousing tools and PowerDesinger data modelling software with business intelligence tools from MicroStrategy, all pre-installed on an IBM Power Systems server.

Many business leaders are struggling to cope with "exploding data volumes", said David Jacobson, senior director of marketing, Sybase; existing data warehouses buckle under the strain of processing such volumes, he added.

"Sybase Analytic Appliance [can] alleviate analytics workloads in a cost-effective manner that is also easy to deploy and maintain," he said in a statement.

The Sybase Analytic Appliance, available immediately comes in three models: Series 100, Series 200 and Series 300. Sybase estimates that the cost of the appliance will average £14k per terabyte.

Rosalie Marshall, IT Week, Tuesday 5 August 2008 at 13:06:00

EMC's latest midrange system promises flash drives and virtual provisioning

Storage heavyweight EMC has confirmed that the latest versions of its Clariion midrange storage systems is ready to ship.

The Clariion CX4 Series will make it easier for customers to rapidly deploy greater volumes of storage capacity. The CX4 range will include EMC's virtual provisioning technology from October, which allows users to present a nominal storage allocation to applications without having to allocate the physical capacity until it is needed.

The new Clariion systems also feature EMC's UltraFlex technology, which supports both dual-protocol Fibre Channel and iSCSI storage area network (SAN) connections in the same array. It will also allow users to easily connect to future technologies, such as Fibre Channel over Ethernet.

The addition of flash drives in October will provide the CX4 Series with up to 10 times the performance the CX3 series contained, EMC claimed.

Flash drives store data using flash memory, unlike conventional hard drives that use mechanically rotating discs.

Drive spin-down that will autonmatically place inactive drives in sleep mode will be available some time in 2009.

"The CLARiiON CX4 Series is unmatched in the industry when it comes to scalability, features, energy efficiency and ease of use," said David Donatelli, president of EMC's storage division in a statement.

Clariion CX4 Series is comprised of four models: 120, 240, 480 and 960. The numbers correspond to the number of disk drives.

The CX4 Series was first unveiled in June at the firm’s annual customer conference, EMC World.

Gareth Morgan, IT Week, Tuesday 5 August 2008 at 11:29:00

Lenovo's ultra-small laptops to hit the shelfs this autumn

Chinese hardware maker Lenovo will release its first range of mini-laptops this autumn, becoming the latest vendor to try an take a slice of this white-hot market.

Lenovo's IdeaPad S9 and S10 netbooks, based on Intel's Atom processor, will offer up to 1GB of memory and either a 160GB hard drive, or 4GB of solid state storage. They will also be available with a choice of Windows XP or Linux.

The mini notebooks "feature the best of basic computing functions in an extremely compact and affordable form," said Liu Jun, senior vice president of Lenovo's consumer business group, in a statement.

The S9 features a 8.9in screen, while the S10 has a 12.2in display. Both are available in a range of colours, including white, black, blue, red and pink.

The IdeaPads are due to ship in early October 2008. The IdeaPad S9 will cost from £279, the S10 from £319.

Lenovo has become the latest computer maker, along with Asus, Acer and HP, looking to exploit the demand for low-cost, lightweight, basic laptops.

Phil Muncaster, IT Week, Tuesday 5 August 2008 at 00:15:00

SC releases new product to help firms improve their security and compliance efforts

Enterprise gateway security firm Secure Computing boosted its reporting capabilities today with the launch of a new tool to help firms gain greater insight into their web traffic and improve security, compliance and performance.

Secure Web Reporter is comprised of the firm's SmartReporter and Content Reporter tools and enables administrators to view enterprise data, such as web surfing and downloading activity and amount of malware blocked, in real-time.

With this information, firms can then change security policies and investigate potential problems, according to Secure's Mike Smart.

"There's been a massive surge in threats targeted at the web," he said. "A lot of people are struggling to make sense of it – but this tool helps you to do this."

The Premium version of the product also features distributed reporting functionality to help spread the reporting burden throughout an organisation, and the ability to produce highly configured reports, said Smart.

In related news security vendor Kaspersky Lab has expanded its hosted offering to include web and instant messaging security.

Kaspersky Hosted Security Services previously contained just anti-spam and anti-virus capabilities.

Dave Bailey, IT Week, Monday 4 August 2008 at 17:52:00

Juniper updates its line of unified access control tools

Juniper Networks has refreshed its line up of its unified access control (UAC) tools, adding new server hardware and firmware upgrades.

Juniper's new Network and Security Manager is based on its existing NetScreen Security Manager, adding support for its J-Series services routes, EX-Series switches, secure sockets layer (SSL) connections and UAC.

The ability it manage SSL and UAC from a single panel will ease the workload of security staff, said Anton Grashion, Juniper's security strategist in Europe, the Middle East and Africa.

"You could have an access control policy for people working remotely or locally," he said.

Juniper has also launched two new UAC management systems, the Infranet Controller (IC) 4500 and the 6500, which allow businesses to implement security policy controls from a central server. The IC 6500 is targeted at large, multinational organisations and can support 30,000 devices simultaneously.

Juniper's UAC 2.2 firmware is built on the open, standards-based Trusted Computing Group's (TCG) Trusted Network Connect (TNC).

With its latest release, Juniper had also added support for Microsoft's Windows Statement of Health (SoH) protocol and its embedded network access protection (NAP) agent, so any Windows Vista users or Windows XP users with service pack 3 installed, could have their network access managed through Juniper Infranet Controllers.

Juniper's UAC 2.2 firmware and the IC 4500 and IC 6500 platforms are available now. UAC 2.2 software is free for customers with current maintenance contracts, but lists at $1,500 a 25 simultaneous endpoint device.

Dave Bailey, IT Week, Monday 4 August 2008 at 14:32:00

European firms will trial Dell's new HPC systems this autumn

IT systems giant Dell will launch a high performance compute (HPC) cluster pilot this autumn.

Dell will unveil a pre-configured range of HPC systems, with between four and 32 nodes, based on the combination of its server and storage hardware with open source software.

The systems will be based on Dell's PowerEdge M-Series blades, with initial system roll outs running under Linux operating systems like Red Hat Enterprise Linux. Dell added it would introduce a Windows-based alternative by the end of the year.

The systems will be trialled at a number of European firms, including UK provider of high street foreign currency, No 1 Currency.

Dell's enterprise solutions director James Quarles said that its pilot programme was designed to provide the compute power needed to accelerate the research process, reduce costs and simplify deployment of HPC systems, allowing European organisations to extend HPC capabilities departmentally.

Rosalie Marshall, IT Week, Monday 4 August 2008 at 12:43:00

Communications Managers Association to show how IT can cut carbon emissions

Trade body the Communications Management Association (CMA) has urged IT leaders to make technology the cornerstone of organisational efforts to cut carbon emissions.

The CMA has joined forces with the Carbon Trust to develop best practice guidelines on how businesses can use information communication technology, such as tele-conferencing and home working, to realise carbon savings.

A joint project, called Carbon Intent, will include a thorough review of all existing research on energy saving strategies businesses have been employing and highlight companies that deploy green products, systems and business processes efficiently.

“Through the Carbon Intent initiative we propose to draw together and share examples of current best practice from video-conferencing to logistics and transport planning to home working,” said Glenn Powell, CMA chief executive, in a statement. "The aim is to create industry-wide guidelines and benchmarks that any ICT professional anywhere can tap into for assistance.”

Daniel Robinson, IT Week, Monday 4 August 2008 at 05:00:00

Chipmaker provides glimpse of next-generation multi-core architecture

Intel has disclosed more details of its forthcoming Larrabee chip architecture that will use multiple processing cores to boost processing speeds in applications such as 3D graphics and scientific and engineering simulations.

Due to ship in 2009 or 2010, the first Larrabee chip will target the personal computer graphics market, Intel said. The firm will present a paper on the architecture at the SIGGRAPH 2008 computer graphics conference in Los Angeles next week.

Larrabee is based on processor cores with the same x86 architecture used in PC chips, but with enhancements such as vector processing and dedicated hardware for functions such as processing graphics textures.

It is designed to be more flexible and programmable than the graphics processor unit (GPU) chips that power today's graphics adapters, according to Intel, while keeping the familiarity and ease of programming of the Intel architecture.

Although each Larrabee core is said to be based on technology from the Pentium, it has been enhanced with features such as multi-threading and 64bit extensions. Intel said that Larrabee's native programming model supports highly parallel applications, and will enable development of graphics APIs and new graphics algorithms as well as general purpose computation.

Rosalie Marshall, IT Week, Friday 1 August 2008 at 16:02:00

Yahoo new shareholder Carl Icahn will not attend firm's annual meeting

The day before Yahoo’s annual shareholder meeting, activist shareholder Carl Icahn announced in his blog he would not be attending and described his new appointment on the Yahoo board as an unhappy compromise.

Icahn agreed to join Yahoo's board 10 days ago, having previously attempted to replace it altogether with a team more compliant to a Microsoft acquisition.

If Icahn’s proxy fight had been successful, the annual meeting would have seen Icahn’s nominated board candidates stand in an election, a negotiation with Microsoft on the cards and the likely dismissal of Yahoo’s chief executive, Jerry Yang.

But Icahn blamed “today’s corporate governance system” for his failure to overthrow the board. “Where large mutual funds control so much of the stock, it is extremely difficult to oust an entire board, no matter how strongly a large number of shareholders feel about the board’s previous actions,” he said.

Icahn said he had joined the board as a last resort. “Realising I could not gain control, I saw no point in spending the final two weeks in a debilitating fight, where little would be accomplished except to build animosity between both caps”.

Icahn will now miss any disappointment aired by Yahoo shareholders on how the board handled the Microsoft purchase offer. His argument for not attending the meeting was because it would turn it into “a media event for no purpose”.

Phil Muncaster, IT Week, Friday 1 August 2008 at 15:04:00

McAfee prepares £23 million deal for DLP vendor Reconnex

Web security firm McAfee has sought to expand its outbound security capabilities with the acquisition of data loss prevention (DLP) vendor Reconnex.

The proposed £23 million cash deal will give McAfee technology which can proactively identify which information in an organisation should be kept confidential and who should have access to it.

The deal follows the purchase of DLP firm Onigma by McAfee last year, and comes after a flurry of recent acquisitions in the space by rivals Symantec and Trend Micro.

“The growing number of high-profile incidents in which customer records, confidential information and intellectual property were leaked, lost or stolen has created an explosive demand for [DLP] solutions,” said IDC analyst Brian Burke, in a statement. “This acquisition creates a great opportunity for McAfee to fulfill a critical need in this space.

IDC predicts the market for these types of solutions, which it has termed information protection and control (IPC) to grow by 33 per cent a year to $1.6 billion in 2011.

Gareth Morgan, IT Week, Friday 1 August 2008 at 13:24:00

Big Blue to build £180m datacentre to host cloud computing services

IBM has confirmed it plans to start offering cloud computing services, which will be housed in a £180m datacentre it is building in North Carolina.

The new facility will also be the testbed for IBM's green datacentre plans, and will be the first one to incorporate its state-of-the-art design principles aimed at making IT more energy efficient.

"This new datacentre is part of IBM's commitment to construct the world's most advanced datacentres," said Bob Greenberg, general manager of IT optimization, in a statement. "We open for business in late 2009."

The new facility will replace existing buildings on IBM's Research Triangle Park site in North Carolina with a 60,000 datacentre, packed with energy-efficient kit and designed to allow additional modules to be built on.

The building will use a mixture of air-, water- and fresh-air-cooling to ensure an optimum operating temperature, delivered in the most energy-efficient manner.

IBM has yet to confirm what services it will offer from the facility.

Rosalie Marshall, IT Week, Thursday 31 July 2008 at 17:42:00

Oracle aims to build up its offerings in software training

Oracle will acquire Global Knowledge Software for an undisclosed sum, to boost it software training offerings.

Global Knowledge Software (GKS) produces programmes that train users on different business software, including that from Oracle, Microsoft and SAP.

“Our customers are looking for a consistent, cost effective training solution across the enterprise to speed software adoption by end-users,” said Ed Abbo, Senior Vice President of Oracle Application Development.

GKS’ products will complement Oracle’s existing training software, Oracle Tutor, along with its learning management software, iLearning and eLearning. Oracle intends to form a global sales unit to extend all its software training offerings across the enterprise.

The transaction is expected to close in the third calendar quarter of 2008.

Phil Muncaster, IT Week, Thursday 31 July 2008 at 17:37:00

Confusing term is misleading IT buyers,according to WCM vendor

Enterprise content management (ECM) is an inaccurate term causing buyer confusion and needs to be "blown apart into its constituent parts" according to web content management firm Interwoven.

Speaking exclusively to IT Week, Interwoven's vice president of Europe, James Murray, argued that the term ECM covers such a broad range of technologies that it lacks any real meaning. The industry should instead revert to talking about its individual components such as document management, search, and records management, he added.

"ECM is poorly understood and our [part of it] is poorly understood – it's time for ECM to be blown apart into its constituent parts because it's confusing for customers," Murray argued.

He added that customers are often misled by the term ECM when applied to technology products, thinking that it will take care of all their content management problems, when in fact an ECM solution is likely to only address a small part of their requirements.

Rosalie Marshall, IT Week, Thursday 31 July 2008 at 17:07:00

Gartner rubbishes OLPC claims that it will produce a $100 laptop within three years

Analyst group Gartner has poured scorn on claims by the One Laptop Per Child group, that it will be able to produce a laptop costing £50 ($100) within the next three years. But Gartner does acknowledge that prices for portable computers are set to tumble.

Prices will be driven down by the combination of demand and declining component costs, said Gartner analyst Annette Jump. But those costs are not falling fast enough to make a $100 laptop a viable proposition within three years, she noted.

Gartner's latest market figures show sales of laptops in Western Europe grew by 45 per cent year-on-year for the second quarter of 2008.

The cheapest laptop in the market, according to a cost breakdown from Gartner, is the OLPC Device. “The OLPC-XO device is slightly cheaper than the other products because of lower processor and motherboard cost, and battery and packaging cost,” noted Jump’s report. Closely following the OLPC-XO in cost is Intel’s Classmate PC and the Asus Eee PC.

Nevertheless, OLPC remains adamant that its target of developing a $100 laptop is on track. “OLPC is going to bring the cost down to a lot less than $100 much sooner than three years,” a company spokesman told IT Week.

Rosalie Marshall, IT Week, Thursday 31 July 2008 at 14:55:00

SAP admits some customers will not benefit from rising support costs

SAP has admitted that its plans to force its customers on to new support contracts will result in many of them paying for services they do not need.

The software maker has angered many of its users by insisting on changes to its support contracts.

Earlier this month, SAP confirmed customers on its Standard Support contracts would be transitioned to Enterprise Support agreement, which provides a more comprehensive level of support.

The UK User Group chairman Alan Bowling argued the new offering was “a one size fits all strategy for support”, and that many users would not use the extensive support package they would be paying nearly 30 per cent more for.

But SAP has no intention to add more flexibility to the support offering, even if it means users paying for services they do not need.

Bill Wohl, SAP vice president of product technology group communications, acknowledged not all customers would benefit from the new offering. “Is it perfect for 100 per cent of our customers? No, but neither is our software. We will go ahead with the programme as planned.”

Wohl maintained that the majority of its user base would benefit from the changes.

“Our job is to be broad enough to help the majority of our customers,” he said. Most customers need a level of support that addresses expanding IT complexity and increased adoption of a service-oriented-architecture (SOA) strategy.

With Enterprise Support, which SAP calls “proactive risk reduction”, customers will be given guidance on how to improve business processes and how an SOA strategy can be successfully delivered.

Wohl added the remaining customers that did not need the next generation of support now would do in years to come. “Our job is to provide support offerings whether they [SAP customers] need it now or in the future,” he added.

But users remain unconvinced by these arguments. “The fact still remains that our members are unhappy about the mandatory nature of this change,” said SAP UK User Group chairman, Alan Bowling.

OCZ breaks into bargain market with 'sub-$100' 32GB Onyx SSD originally appeared on Engadget on Wed, 10 Mar 2010 16:15:00 EST. Please see our terms for use of feeds.

Continue reading Google Maps adds bicycle directions, turns out its nothing like 'Quicksilver'

Google Maps adds bicycle directions, turns out its nothing like 'Quicksilver' originally appeared on Engadget on Wed, 10 Mar 2010 15:52:00 EST. Please see our terms for use of feeds.

HTC HD2 said to be launching March 24th for $199, no Windows Phone 7 support originally appeared on Engadget on Wed, 10 Mar 2010 15:31:00 EST. Please see our terms for use of feeds.

OnLive Game Service to launch on June 17 in the US for $15 a month originally appeared on Engadget on Wed, 10 Mar 2010 15:10:00 EST. Please see our terms for use of feeds.

Continue reading Computing pioneer Chuck Thacker wins Turing Award

Computing pioneer Chuck Thacker wins Turing Award originally appeared on Engadget on Wed, 10 Mar 2010 14:51:00 EST. Please see our terms for use of feeds.

HTC Legend already trickling into customers' waiting hands? originally appeared on Engadget on Wed, 10 Mar 2010 14:16:00 EST. Please see our terms for use of feeds.

Continue reading Google Apps Marketplace opens for business, for business

Google Apps Marketplace opens for business, for business originally appeared on Engadget on Wed, 10 Mar 2010 13:53:00 EST. Please see our terms for use of feeds.

Gallery: HTC Legend review

Continue reading HTC Legend review

HTC Legend review originally appeared on Engadget on Wed, 10 Mar 2010 13:20:00 EST. Please see our terms for use of feeds.

Boeing announces compact, energy-efficient 3D camera originally appeared on Engadget on Wed, 10 Mar 2010 12:59:00 EST. Please see our terms for use of feeds.

Continue reading Panasonic 3D home entertainment system goes on sale to a country in mourning

Panasonic 3D home entertainment system goes on sale to a country in mourning originally appeared on Engadget on Wed, 10 Mar 2010 12:37:00 EST. Please see our terms for use of feeds.

Gallery: Casio Exilim G EX-G1 hands-on

Continue reading Casio Exilim G EX-G1 review

Casio Exilim G EX-G1 review originally appeared on Engadget on Wed, 10 Mar 2010 12:04:00 EST. Please see our terms for use of feeds.

Continue reading Target launches first scannable mobile coupon program, frugalistas going wild

Target launches first scannable mobile coupon program, frugalistas going wild originally appeared on Engadget on Wed, 10 Mar 2010 11:42:00 EST. Please see our terms for use of feeds.

Continue reading G-Tec Intendix brain-computer interface ready for consumers (video)

G-Tec Intendix brain-computer interface ready for consumers (video) originally appeared on Engadget on Wed, 10 Mar 2010 11:19:00 EST. Please see our terms for use of feeds.

Gallery: Lenovo IdeaPad S10-3t review

Continue reading Lenovo IdeaPad S10-3t review

Lenovo IdeaPad S10-3t review originally appeared on Engadget on Wed, 10 Mar 2010 10:52:00 EST. Please see our terms for use of feeds.

KDDI concocts snooping mobile phones, line managers rub hands with glee originally appeared on Engadget on Wed, 10 Mar 2010 10:27:00 EST. Please see our terms for use of feeds.

Continue reading How the Terminator's .45 Longslide with laser sighting came to be (video)

How the Terminator's .45 Longslide with laser sighting came to be (video) originally appeared on Engadget on Wed, 10 Mar 2010 10:03:00 EST. Please see our terms for use of feeds.

US mineral companies to tech industry: drill, baby, drill originally appeared on Engadget on Wed, 10 Mar 2010 09:44:00 EST. Please see our terms for use of feeds.

Continue reading Combustible carbon nanotubes give off electricity, make really tiny fires (video)

Combustible carbon nanotubes give off electricity, make really tiny fires (video) originally appeared on Engadget on Wed, 10 Mar 2010 09:22:00 EST. Please see our terms for use of feeds.

Continue reading Robosoft Kompai takes care of your elderly so you don't have to (video)

Robosoft Kompai takes care of your elderly so you don't have to (video) originally appeared on Engadget on Wed, 10 Mar 2010 09:01:00 EST. Please see our terms for use of feeds.

Korea starts testing 'recharging road,' might make it part of its public transport system originally appeared on Engadget on Wed, 10 Mar 2010 08:42:00 EST. Please see our terms for use of feeds.

Continue reading Motion-Sim 4DOF racing simulator will take your retirement fund for the ride of its life (video)

Motion-Sim 4DOF racing simulator will take your retirement fund for the ride of its life (video) originally appeared on Engadget on Wed, 10 Mar 2010 08:18:00 EST. Please see our terms for use of feeds.

HP MediaSmart add-in adds TiVo compatibility, enables video transfers to and fro originally appeared on Engadget on Wed, 10 Mar 2010 07:59:00 EST. Please see our terms for use of feeds.

CE-Oh no he didn't!: Katzenberg says 'beautifully styled' 3D glasses won't make you look like a dweeb originally appeared on Engadget on Wed, 10 Mar 2010 07:32:00 EST. Please see our terms for use of feeds.

Gallery: OO High Definition Wireless Projector Concept

Continue reading OO HD wireless projector concept reaches for the stars, almost grabs 'em (video)

OO HD wireless projector concept reaches for the stars, almost grabs 'em (video) originally appeared on Engadget on Wed, 10 Mar 2010 07:05:00 EST. Please see our terms for use of feeds.

Continue reading Happy Hour watches open bottles, doors unto new worlds (video)

Happy Hour watches open bottles, doors unto new worlds (video) originally appeared on Engadget on Wed, 10 Mar 2010 06:34:00 EST. Please see our terms for use of feeds.

FCC will consider 'free or very low cost wireless broadband' service originally appeared on Engadget on Wed, 10 Mar 2010 06:13:00 EST. Please see our terms for use of feeds.

Over 50 ARM-based tablets launching this year originally appeared on Engadget on Wed, 10 Mar 2010 05:38:00 EST. Please see our terms for use of feeds.

Continue reading Steampunk sequencer generates audio from Lego blocks

Steampunk sequencer generates audio from Lego blocks originally appeared on Engadget on Wed, 10 Mar 2010 05:18:00 EST. Please see our terms for use of feeds.

Martin Jetpack priced at $86,000, mere mortals will soon be able to buy one too originally appeared on Engadget on Wed, 10 Mar 2010 04:24:00 EST. Please see our terms for use of feeds.

MIT gurus use polyethylene to suck heat away from your next CPU originally appeared on Engadget on Wed, 10 Mar 2010 03:26:00 EST. Please see our terms for use of feeds.

Continue reading Bluetooth Access Point brings text messaging, voice messaging, email to the friendly skies

Bluetooth Access Point brings text messaging, voice messaging, email to the friendly skies originally appeared on Engadget on Wed, 10 Mar 2010 02:19:00 EST. Please see our terms for use of feeds.

Gallery: EA Sports Active 2.0 accessories at GDC 2010

Continue reading EA Sports introduces Active 2.0 at GDC, complete with sensors galore

EA Sports introduces Active 2.0 at GDC, complete with sensors galore originally appeared on Engadget on Wed, 10 Mar 2010 01:04:00 EST. Please see our terms for use of feeds.

Gallery: Dell Streak leaked internal documents

Leaked Dell Streak flyer shows multitude of color options, confirmed specifications originally appeared on Engadget on Tue, 09 Mar 2010 23:53:00 EST. Please see our terms for use of feeds.

Pentax gets official with 40 megapixel 645D medium format camera originally appeared on Engadget on Tue, 09 Mar 2010 23:38:00 EST. Please see our terms for use of feeds.

Continue reading MSI starts shipping two 12.1-inch, AMD-powered Wind12 U230 netbooks

MSI starts shipping two 12.1-inch, AMD-powered Wind12 U230 netbooks originally appeared on Engadget on Tue, 09 Mar 2010 22:37:00 EST. Please see our terms for use of feeds.

Continue reading Razer and Sixense distribute SDK and FPS shooter utility through Steam

Razer and Sixense distribute SDK and FPS shooter utility through Steam originally appeared on Engadget on Tue, 09 Mar 2010 21:55:00 EST. Please see our terms for use of feeds.

Gallery: Motorola CLIQ XT with MotoBlur hands-on

Continue reading Motorola CLIQ XT hands-on

Motorola CLIQ XT hands-on originally appeared on Engadget on Tue, 09 Mar 2010 21:10:00 EST. Please see our terms for use of feeds.

Gallery: Windows Phone 7 Series XNA screenshots

Continue reading Microsoft shows off XNA games running on Windows Phone, full 3D is a go

Microsoft shows off XNA games running on Windows Phone, full 3D is a go originally appeared on Engadget on Tue, 09 Mar 2010 20:50:00 EST. Please see our terms for use of feeds.

Continue reading Entelligence: Aiming high or another Mylo?

Entelligence: Aiming high or another Mylo? originally appeared on Engadget on Tue, 09 Mar 2010 20:23:00 EST. Please see our terms for use of feeds.

Does Android dream of DIY cushions? originally appeared on Engadget on Tue, 09 Mar 2010 19:44:00 EST. Please see our terms for use of feeds.

In response to Mark Callaghan’s post mk-schema-change.

I apologize for not commenting on the post itself, I do not hold a Facebook account. Anyway this is a long write, so it may as well deserve a post of its own.

Some of the work Mark is describing already exists under openark kit’s oak-online-alter-table. Allow me to explain what I have gained there, and how the issue can be further pursued. There is relevance to Mark’s suggestion.

oak-online-alter-table uses a combination of locks, chunks and triggers to achieve an almost non-blocking ALTER TABLE effect. I had a very short opportunity to speak with Mark on last year’s conference, in between bites. Mark stated that anything involving triggers was irrelevant in his case.

The triggers are a pain, but I believe a few other insights from oak-online-alter-table can be of interest.

The first attempt

My first attempt with the script assumed:

• Table has an AUTO_INCREMENT PRIMARY KEY column
• New rows always gain ascending PRIMARY KEY values
• PRIMARY KEY never changes for an existing row
• PRIMARY KEY values are never reused
• Rows may be deleted at will
• No triggers exist on the table
• No FOREIGN KEYs exist on the table.

So the idea was: when one wants to do an ALTER TABLE:

1. Create a ghost table with the new structure.
2. Read the minimum and maximum PK values.
3. Create AFTER INSERT, AFTER UPDATE, AFTER DELETE triggers on the original table. These triggers will propagate the changes onto the ghost table.
4. Working out slowly, and in small chunks, copy rows within recorded min-max values range into the ghost table. The interesting part is where the script makes sure there’s no contradiction between these actions and those of the triggers, (whichever came first!). This is largely solved using INSERT IGNORE and REPLACE INTO in the proper context.
5. Working out slowly and in chunks again, we remove rows from the ghost table, which are no longer existent in the original table.
6. Once all chunking is complete, RENAME original table to *_old, and ghost table in place of the original table.

Steps 4 & 5 are similar in concept to transactional recovery through redo logs and undo logs.

The next attempt

Next phase removed the AUTO_INCREMENT requirement, as well as the “no reuse of PK”. In fact, the only remaining constraints were:

• There is some UNIQUE KEY on the table which is unaffected by the ALTER operation
• No triggers exist on the table
• No FOREIGN KEYs exist on the table.

The steps are in general very similar to those listed previously, only now a more elaborate chunking method is used with possible non-integer, possible multi-column chunking algorithm. Also, the triggers take care of changes in UNIQUE KEY values themselves.

mk-schema-change?

Have a look at the wiki pages for OnlineAlterTable*. There is some discussion on concurrency issues; on transactional behavior, which explains why oak-online-alter-table performs correctly. Some of these are very relvant, I believe, to Mark’s suggestion. In particular, making the chunks copy; retaining transactional integrity, etc.

To remove any doubt, oak-online-alter-table is not production ready or anywhere near. Use at your own risk. I’ve seen it work, and I’ve seen it crash. I got little feedback and thus little chance to fix things. I also didn’t touch the code for quite a few months now, so I’m a little rusty myself.

1. Create the new table on the master. The new table might use MyISAM without indexes initially to make the insert as fast as possible and reduce the load on InnoDB.
2. Run set sql_log_bin=0 as what follows should not be written to the binlog
3. Run start transaction with consistent innodb snapshot to start an Innodb transaction and get current binlog offset of the master
4. Run insert into new_table select * from original_table on the master. Alas, this will get a transaction duration read lock on every row in original_table unless you use row based replication or hack InnoDB or set innodb_locks_unsafe_for_binlog.
5. Convert new_table to InnoDB and create indexes on it
6. Replay changes from the binlogs after the point in time recorded in step #3. This should extract changes to original_table and replay them against new_table.

Together with Berndt I’ll be presenting on the new features in MySQL Cluster 7.1 at this year’s MySQL Cluster User Conference – Santa Clara, on April 12th. If you’re interested in using MySQL Cluster but aren’t sure how to get started (or you’ve used it but would like some tips) then this is a great opportunity. Check out the presentation description.

If you register by 15 March then you get the early-bird price and if you use this ‘friend of a speaker’ code then you get an additional 25% off: mys10fsp

If you missed the recent webinar on running MySQL Cluster on Windows then you can watch/listen to the replay at http://www.mysql.com/news-and-events/on-demand-webinars/display-od-517.html

Working on mycheckpoint, I have the intention of adding custom monitoring. That is, letting the user define things to monitor. I have my own thoughts, I would be grateful to get more input!

What would the user want to monitor?

Monitoring for the number of SELECT statements per second, InnoDB locks, slave replication lag etc. is very important, and monitoring utilities provide with this information. But what does that tell the end user? Not much.

The experienced DBA may gain a lot. The user would be more interested in completely other kind of information. In between, some information is relevant to both.

Say we were managing an on-line store. We want to monitor the health of the database. But the health of the database is inseparable from the health of the application. I mean, having little to no disk usage is fine, unless… something is wrong with the application, which leads to no new purchases.

And so a user would be interested in monitoring the number of purchases per hour, or the time passed since last successful purchase. This kind of data can only be generated by a user’s specific query. Looking at the charts, the user would then feel safer and confident in the wellness of his store app.

But let’s dig further. We want the store’s website to provide with good response. In particular, the query which returns the items in a customer’s cart must react quickly. Our user would not only want to see that purchases get along, but also that page load times (as in our example) are quick for those critical parts. And so a user should be able to monitor the time it took to execute a given query.

It can be of further interest to know how many times per second a given query is executed. This part is not easily done on the server side, and requires the user’s cooperation (or else we must analyze the general log, sniff, or set up a proxy). If the user is willing, she can log to some table each time she executes a certain query. Then we’re back to monitoring a regular table, as with the first example.

It is also possible to monitor for a query’s execution plan. Is it full scan? How many rows are expected? But given that we can monitor the time it took to execute a query, I’m not sure this is useful. If everything runs fast enough — who cares about how it executes?

Some of the above can be monitored on an altogether higher level: if  we’re talking about some web application, then we can use our Apache logs to determine load time for pages, or number of requests to our “cart items” page. But not always do we work with web servers, and we may be interested in checking the specific queries behind the scenes.

Summary

Custom monitoring can include:

• User defined queries (number of concurrent visitors; count of successful operations per second; number of rows per given table or condition; …)
• Execution time for user defined queries (time it takes to return cart items; find rows matching condition; sort a table; …)
• Number of executions for a given query, per second.

I intend to incorporate the above into mycheckpoint as part of its standard monitoring scheme.

Please share your thought below.

CREATE TEMPORARY TABLE test.proc_stat (
  seq tinyint auto_increment primary key, 
  the_key char(25) NOT NULL, 
  user bigint,
  nice bigint, 
  system bigint,
  idle bigint, 
  iowait bigint,
  irq bigint,
  softirq bigint, 
  steal bigint, 
  guest bigint, 
  other bigint
);  

/* MySQL treats consecutive delimiters as separate fields, so some fancy footwork
   is required to load the file successfully.  The file includes a cpu field followed
   by two spaces which is the sum of all the individual CPUs in the system.  

   To account for this each row is read into some MySQL variables. Those variables 
   are examined to determine which field holds the correct value.
*/
LOAD DATA INFILE '/proc/stat' 
   IGNORE INTO TABLE test.proc_stat 
   FIELDS TERMINATED BY ' ' 
   (@the_key, @val1, @val2, @val3, @val4, @val5, @val6, @val7, @val8, @val9, @val10)
SET other = IF(@the_key like 'cpu%', NULL, @val1),  
    the_key = @the_key, 
    user = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val1, 0), IFNULL(@val2,0))),
    nice = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val2, 0), IFNULL(@val3,0))),
  system = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val3, 0), IFNULL(@val4,0))),
    idle = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val4, 0), IFNULL(@val5,0))),
  iowait = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val5, 0), IFNULL(@val6,0))),
     irq = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val6, 0), IFNULL(@val7,0))),
 softirq = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val7, 0), IFNULL(@val8,0))),
   steal = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val8, 0), IFNULL(@val9,0))),
   guest = IF(@the_key NOT LIKE 'cpu%', NULL, IF(@the_key != 'cpu', IFNULL(@val9, 0), IFNULL(@val10,0)));

mysql> select * from test.proc_stat;
+-----+---------------+--------+-------+--------+------------+--------+------+---------+-------+-------+------------+
| seq | the_key       | user   | nice  | system | idle       | iowait | irq  | softirq | steal | guest | other      |
+-----+---------------+--------+-------+--------+------------+--------+------+---------+-------+-------+------------+
|   1 | cpu           | 378340 | 33588 |  82489 | 1838257830 |  75444 |  750 |   23065 |     0 |     0 |       NULL |
|   2 | cpu0          |   4152 |   125 |   1613 |  114920899 |    624 |    0 |     869 |     0 |     0 |       NULL | 
|   3 | cpu1          |   2182 |    78 |   1474 |  114924477 |     50 |    2 |       3 |     0 |     0 |       NULL | 
|   4 | cpu2          |   6037 |  5418 |   2289 |  114914024 |     55 |   34 |     401 |     0 |     0 |       NULL | 
|   5 | cpu3          |   3519 |    55 |    842 |  114923794 |     37 |    1 |       1 |     0 |     0 |       NULL | 
|   6 | cpu4          |  71851 |  5443 |   6656 |  114840363 |   3197 |   11 |     720 |     0 |     0 |       NULL | 
|   7 | cpu5          |   2435 |     5 |    801 |  114924963 |     29 |    2 |       0 |     0 |     0 |       NULL | 
|   8 | cpu6          | 136246 |  4711 |  36628 |  114690032 |  46119 |   20 |   14471 |     0 |     0 |       NULL | 
|   9 | cpu7          |   1119 |     2 |    366 |  114926691 |     40 |    1 |       0 |     0 |     0 |       NULL | 
|  10 | cpu8          |   4126 |    34 |   2772 |  114920032 |     92 |    1 |    1153 |     0 |     0 |       NULL | 
|  11 | cpu9          |   1618 |     2 |    694 |  114925811 |     77 |    1 |       0 |     0 |     0 |       NULL | 
|  12 | cpu10         |  18096 |  8735 |   6823 |  114891588 |    396 |  179 |    2379 |     0 |     0 |       NULL | 
|  13 | cpu11         |   7243 |  2583 |   3559 |  114914559 |    241 |    1 |       2 |     0 |     0 |       NULL | 
|  14 | cpu12         |   5215 |  2380 |   2776 |  114915814 |    417 |  342 |    1237 |     0 |     0 |       NULL | 
|  15 | cpu13         |   3224 |    28 |   1507 |  114923336 |     77 |    2 |       0 |     0 |     0 |       NULL | 
|  16 | cpu14         | 109818 |  3979 |  13071 |  114775431 |  23901 |  143 |    1823 |     0 |     0 |       NULL | 
|  17 | cpu15         |   1450 |     1 |    612 |  114926010 |     83 |    1 |       0 |     0 |     0 |       NULL | 
|  18 | intr          |   NULL |  NULL |   NULL |       NULL |   NULL | NULL |    NULL |  NULL |  NULL | 1176485951 | 
|  19 | ctxt          |   NULL |  NULL |   NULL |       NULL |   NULL | NULL |    NULL |  NULL |  NULL |  171220339 | 
|  20 | btime         |   NULL |  NULL |   NULL |       NULL |   NULL | NULL |    NULL |  NULL |  NULL | 1267061074 | 
|  21 | processes     |   NULL |  NULL |   NULL |       NULL |   NULL | NULL |    NULL |  NULL |  NULL |     168510 | 
|  22 | procs_running |   NULL |  NULL |   NULL |       NULL |   NULL | NULL |    NULL |  NULL |  NULL |          1 | 
|  23 | procs_blocked |   NULL |  NULL |   NULL |       NULL |   NULL | NULL |    NULL |  NULL |  NULL |          0 | 
+-----+---------------+--------+-------+--------+------------+--------+------+---------+-------+-------+------------+
23 rows in set (0.00 sec)

# Time: 100309 18:48:23
# User@Host: root[root] @ localhost []
# Query_time: 0  Lock_time: 0  Rows_sent: 1  Rows_examined: 1

# Query_time: 0  Lock_time: 0  Rows_sent: 1  Rows_examined: 1\
           Thread_id: 3 Errno: 0 Start: 18:48:23 End: 18:48:23

# Time: 100310  7:51:28
# User@Host: root[root] @ localhost []
# Query_time: 0  Lock_time: 0  Rows_sent: 1  Rows_examined: 0 Thread_id: 1 Errno: 0 \
Killed: 0 Bytes_received: 104 Bytes_sent: 161 Read_first: 0 Read_last: 0 Read_key: 0 \
Read_next: 0 Read_prev: 0 Read_rnd: 0 Read_rnd_next: 0 Sort_merge_passes: 0 \
Sort_range_count: 0 Sort_rows: 0 Sort_scan_count: 0 Created_tmp_disk_tables: 0 \
Created_tmp_tables: 0 Start:  7:51:28 End:  7:51:28

I'll be presenting "The Thinking Person's Guide to Data Warehouse Design" at the upcoming MySQL User conference. While a lot of people think that bad SQL code is the #1 wrecking ball of data warehouses and marts, the fact is that poor database design is the first cause of both downtime and bad performance. In my presentation, I'll do my best to show how up-front worRead More...
PlanetMySQL Voting: Vote UP / Vote DOWN

Video fragments of my session posted at the end — read on.

I arrived at Omni Mandalay Hotel on Sunday evening with Dan Norris. I was flying through Chicago and it turned out that Dan was on the same flight and only few rows behind me. Small world.

Preparations for the conference were very chaotic on my part and, of course, I didn’t have either of my presentations ready. I was very stressed and getting sick as well — it looked like a complete disaster waiting to happen. I’d like to say that I was feeling like Doug Burns as he often managed to get sick just before a conference. Of course, I worked on my slides for the last few days as well as on the flight and presentation was slowly getting there but boy was I tired!

I quickly said hello to the crowd in the bar on the way to my room and rushed away to do some more damage to my slides. And then I had a brilliant idea — I could still see one of my best mates and do something good about my presentation! I asked Doug if he was interested in the preview (he probably wasn’t interested but he couldn’t say it to me) especially that my session wasn’t on his original agenda. Of course, that would mean that he had to leave a bunch of other good friends and spend some time tete-a-tete. Knowing Doug, this is some of the hardest thing to ask from him but it shows how good of a friend he is! (Plus, everyone thinks that he is anti-social anyway. Shhhh!)

Doug has made my day — while he provided lots of ideas and feedback on few things that I was lucking, he generally approved the idea and confirmed that it wasn’t totally crazy. I guess that was all I needed back then and Doug knew how nervous I was about it. (Thanks mate!)

So I called Sunday a day very early and went to bed before midnight. I really needed some sleep. Woken up by the alarm at 5AM (I woke up few times during the night looking at the clock — making sure I didn’t sleep through) and slides were ready just before lunch. I even managed to do a test run and it took 65 minutes — a wee bit too long for one hour session. But it was good test and I knew I had to be just a bit more concise in few parts.

Mi morning was very productive. Unfortunately, I missed the opening keynote from Tom Kyte. Such a pity! If what Doug wrote is true, Tom was talking about the mistakes we make *because* of our experience and our assumptions. This was exactly one of the points I was making in my Battle Against Any Guess — experience is danger. I wish I could see Tom’s example. Oh well, maybe another time.

I managed to attend half of the Richard Foote’s session on indexes but my mind was far away — with my own slides. Though, I did manage to focus on bitmap indexes part and the myth of bitmap indexes not working well for columns with high cardinality. Very interesting conclusions. I’m still wondering how much overhead updates will do to such bitmap index.

After lunch, it was my turn. I ordered few copies of the latest OakTable book — Expert Oracle Practices: Oracle Database Administration from the Oak Table — that I co-authored with the bunch of other Oakies. I contributed chapter 1 in the book titled just like my presentation — Battle Against Any Guess. The plan was to give a copy away during the presentation and do a draw for another one at the end of the session. I was so nervous that I forgot about it until the end of the session so I just did a draw for two copies. The lucky winners were Lynn-Georgia Tesch and Surendra Anchula. Congratulations! For the rest of you who left the contact details — please stay tuned and we’ll organize few things online.

Now the main topic of this post — my presentation. What’s unusual about this session is that it’s not some technical stuff that I usually do but a more conceptual and motivational talk. Could I pull it off? Well, I think it went fairly well in general even though I did identify few rough places and my lack of English language mastering. Might need to work a little bit more on the flow of the presentation.

We had quite a few good laughs. Later, people in the next hall were asking about it and Dan was making the jokes on the stage so it must have been loud. Anyway, I think nobody fell asleep and I managed to get people thinking about the topic. I received many “thank you” notes yesterday and compliments on a good session so by the end of the day I was more and more pleased. Thanks everyone for attending and especially big thanks to those of you who brought to my attention examples from their own battles. If you have more to discuss — contact me by email (my last name) {at} pythian.com.

Thanks to Marco Gralike for recording some fragments and sharing them. I think he has more to come.

This is the introductory couple minutes. You can definitely notice how nervous I am starting on the stage:

Solving the wrong problem example:

That’s all for now. Stay tuned — more to come.

As part of my upcoming FREE my.cnf check advice I first need to ask people to provide the current MySQL configuration file commonly found as a file named my.cnf

If only that question was easy to answer!

Use of configuration files

MySQL will by default use at least one configuration file from the following defaults. MySQL also uses a cascade approach for configuration files. When you have multiple files in the appropriate paths you can see unexpected behavior when you override certain values in different files.

You can however for example specify –no-defaults to use no configuration file, or add options to your command line execution, so even looking at all configuration files is no guarantee of your operating configuration.

However for most environments, these complexities do not exist.

Default Location

By default and on single instance MySQL servers you are most likely to find this file called my.cnf and found at:

• /etc/my.cnf
• /etc/mysql/my.cnf

These are known as the global options files.

Alternative Locations

MySQL has both instance specific and user specific locations. For the inclusion of an instance specific file, the location is:

• $MYSQL_HOME/my.cnf

where MYSQL_HOME is a defined environment variable. Historical MySQL versions also looked at [datadir]/my.cnf however I am unaware if this is applicable in 5.x versions.

You can also specific options on a per user basis for default inclusion. These are found at:

• $HOME/.my.cnf

Distro specific locations

Ubuntu for example also provides an ability to add options via an include directory.

Specifying a configuration at runtime

While you may have these default files, you may elect to start mysql with a specific configuration file as specified by –defaults-file. This option will override all global/instance/user locations and use just this configuration file. You can also specify additional configuration that supplements and not overrides the default with –defaults-extra-file.

What files are on my system?

Again, assuming the default names you can perform a brute force check with:

$ sudo find / -name "*my*cnf"

This is actually worthwhile, especially if you find a /root/.my.cnf file which is default MySQL settings for the Operating System ‘root’ user.

MySQL recommendations

MySQL by default provides a number of recommended files however these are generally outdated especially for newer hardware. These files include my-huge.cnf, my-large.cnf, my-medium.cnf, my-small.cnf and my-innodb-heavy-4G.cnf. Don’t assume replacing your configuration with one of these files will make your system perform better.

MySQL made some attempt to correct these and at least some very poor defaults with MySQL 5.4 however I am unsure what’s in MySQL 5.5

MySQL Configuration at runtime

While several commands can help with identifying your configuration files and print defaults etc, it’s also possible to change your configuration at runtime. It’s possible that these changes are not reflected in your configuration files and pose an additional mismatch.

References

• Using Option Files
• Server Command Options

Data comparison is a difficult and resource-intensive process. For convenience, this process can be divided into several steps.
First, you should compare tables from one database on one server with the database on the other server. You should choose columns for data comparison, and also choose a column that will be a comparison key.
The next step is to choose all data from these tables or some specified part of the data.
The third and the most important step is comparison of the two tables by the selected comparison key itself. During this process the status of each record is set to “only in source”, “only in target”, “different”, or “equal”.
The final steps of the data comparison process are including records to the synchronization and synchronization itself. During these steps records needed for synchronization are chosen, update script is created, and after that the script is executed.
You can read a detailed description of the comparison process here.

Now let’s look at the third step (data comparison) thoroughly.

There are several ways of data comparison that differ only by the side where data comparison is going to be performed – on the server side or on the client PC.

Data comparison on the server side is performed using the resources of the server.
The algorithm of comparison is the following:
1. For each record of each of the two tables its checksum is calculated;
2. Then the checksum of every record from one table is compared to the checksum of the corresponding record from another table and conclusion if the records are equal or different is made;
3. The comparison result is stored in a temporary table on the server.

Performance indicators:
1. The speed of data comparison directly depends on the server capacity and occupancy;
2. The maximal size of database for comparison is limited by the resources of the server itself.

Advantages:
1. There is no need to transfer large amounts of data for comparison to the client PC through network. This way we save network traffic;
2. The speed of comparison does not depend on the client PC resources;
3. Ability to compare blob data of any size.

Disadvantages:
1. Because of the record checksum calculation algorithm in some cases different data can result in equal checksum, and instead of the expected “different” status the “equal” status will be received;
2. There is no flexibility in the synchronization and comparison options usage;
3. There is no possibility to view records differences and exclude a part of the records from the synchronization manually;
4. During the synchronization script creation you should perform data transfer from the server to the client side;
5. The control checksum calculation of a large amount of records consumes all server resources;
6. One should provide extra space on the server for the comparison results storage in the temporary table.

As we can see, this way of comparison has more disadvantages than advantages, that’s why this way is rarely used.

Data comparison on the client PC is performed using the client machine resources, and the server only provides data for comparison. In turn, this way of comparison can be divided into several more ways depending on the way how comparison information will be stored.

Comparing Data on local PC when comparison result is stored in RAM.
The comparison algorithm is the following:
1. Server passes all data from both tables to the local PC;
2. Every record of every table is placed to RAM and is compared without checksum calculation;
3. If a record gets “only in source”, “only in target” or “equal” status, only comparison key is stored in RAM. If records get “different” status, they are placed to RAM for storage completely.

Performance indicators:
1. The speed of data comparison directly depends on the client PC resources and on the speed of data transfer through the network;
2. Maximum size of the database for comparison is limited by the size of RAM on the client PC, and this maximum size also depends on the degree to which the databases that should be compared are different – the smaller is the amount of different records, the larger databases can be compared.

Advantages:
1. Minimal server occupancy – server performs only simple data selection;
2. The simplest algorithm of data comparison because records are sorted on the client side;
3. Flexibility in the comparison options usage;
4. Minimal size of the comparison data store;
5. Status of every record for any data is always correct.

Disadvantages:
1. To view records with “only in source”, “only in target”, or “equal” status an extra data selection is needed;
2. An extra data selection is needed to create a synchronization script;
3. OutOfMemory Exception may be arisen when there are a lot of differences in data in databases;
4. Possibility to compare blob data only of the size that equals to the size of free RAM.

This way of comparison is implemented in dbForge Data Compare for SQL Server v1.10, dbForge Data Compare for MySQL v2.00 and allows to compare databases of any size if data in these databases does not differ a lot.

Comparing Data on local PC when comparison result is stored as a cashed file on the disk.
The algorithm of comparison is the following:
The server passes all data from both tables sorted by comparison key to a local PC. Data is read by bytes, compared without checksum calculation and written to a file on the disk.

Performance indicators:
1. The speed of data comparison directly depends on the client PC resources and on the speed of data transferring through the network;
2. The maximum size of a database to compare is limited by free disk space and does not depend on the degree of data difference in databases.

Advantages:
1. Medium server occupancy – server performs data sorting and selection;
2. To view records and synchronization script creation extra requests to the server are not necessary;
3. The status for every record is always correct for any data;
4. Possibility to compare blob data of the size equal to the size of free space available on the disk.

Disadvantages:
1. Difficult algorithm of data comparison for the records comparison key of which is of the string data type;
2. Difficult algorithm of disk cash for temporary information storage creation.

We can see that in this case the only disadvantage of this way of comparison is the difficulty of implementation. There are more advantages than in the ways of comparison listed above. That’s why this way of comparison will be used in the new version of dbForge Data Compare for SQL Server v2.00 and dbForge Data Compare for MySQL v3.00 for data comparison.

Few weeks ago I was at FOSDEM.  It was really amazing experience. I meet many interesting people, learned quite some thing and I returned full of enthusiasm. Open Source events are really great.

But all the fun wasn't over even after the FOSDEM. I spent few more days in Bruxelles attending MySQL packagers meeting organized by SUN/Oracle. We spent quite some time talking to each other. We learned what MySQL people are doing and how. And they learned how do we deal with MySQL and what is troubling us. And many good things will come from this.

First but certainly not last of them is about to appear now. One very interesting thing we learned at meeting was MySQL release policy. What openSUSE and Ubuntu and maybe some others are doing is that after release date there is generaly no version updates allowed. We are only fixing serious bugs and security related issues. It takes quite some work. What we learned is that new releases in stable branch of MySQL are in fact maintanance updates. If you update from 5.1.43 to 5.1.44 you wouldn't get any new features. All you will get are bugfixes. And only bugfixes of serious or security related issues. Does it sound familiar? Yes it is the same thing we are doing! So I discussed it with our maintanance team. And we came to the conclusion that we want to give our users all serious fixes. Not only these few selected. And the best way to do it is to use maintanance updates provided by MySQL people themself. I'm not saying that I don't have enough confidence to play with MySQL sources, but I think that MySQL people can do it better

Yes, you are guessing right. What I'm trying to say is that we are going to update MySQL to the latest available version. This means 5.1.44 for openSUSE 11.2 and 5.0.90 for older openSUSE. We will start with 11.2 as version gap is smaller there and if everything will proceed smoothly, we will continue with 11.1 and 11.0. For 11.2 you can help by testing update. Currently 5.1.44 update is prepared for 11.2 in server: database: STABLE and I'm running some final tests. If you want, you can try it too (not recomended on production servers yet) and if you'll find any problems, please report them before it will hit official updates.

Remember, this is just the beginning. I've got some bigger plans regarding MySQL in 11.3

Changes (as compared to 8.22) include:

Features:
* Added an option to define a ‘color code’ for a connection. The color will be used as background color in the Object Browser.
* A Query Builder session can now be saved and resumed.
* In Query Builder a table alias can be defined for any table by double-clicking the title bar of the table symbol.
* In RESULT tab results can now be retrieved page-wise. This is ON as default with this build with a defined LIMIT of 1000 rows. For a specific query user can change and for this specific query the setting is persistent across sessions. Also read ‘miscellaneous’ paragraph below.
* Added a context menu to Query Builder canvas.

Bug Fixes:
* Deleting a user would leave non-global privileges orphaned in the ‘mysql’ database. Now we use DELETE USER syntax if server supports.
* Also using EDIT USER dialogue to change host or user specifier for a user would not move non-global privileges. We have split the old ALTER USER dialogue into two: a EDIT USER and RENAME USER dialogue. The latter will use RENAME USER syntax if server supports.
* On Wine Data Sync could generate a malformed XML-string what would case Data Sync to abort.
* Fixed an issue where SSH-tunneling failed with public/private key authentication. Technically the fix is in the PLINK binary shipped with SQLyog.
* SJA failed to send notification mails if Yahoo SMTP servers were used. Note that the fix disables encryption option with Yahoo SMTP servers – but it won’t work anyway due to a non-standard SMTP implementation server-side.
* When importing data from a Universe ODBC-source string data could be truncated.
* The fix in 8.22 for the issue that horizontal scrollbar in GRID would sometime not appear was not complete. It could still happen.
* SQLyog will now trim trailing whitespaces in Connection Manager and Create object dialogs to avoid MySQL Errors..
* Opening a file from ‘recent files’ list could crash SQLyog if a Query Builder or Schema Designer tab was selected and the file specified was not a valid XML file for that tab. This bug was introduced in beta 1.
* When calling a Stored Procedure with more than one SELECT statement from ‘Notifications Services’ only one result set was sent by mail.
* The sja.log file had no line-breaks between what was recorded for two jobs.
* On multi-monitor system resizable dialogues could open on the wrong monitor. New implementation is like this: on multi-monitor systems main program dialogue and ‘first child dialogue’ (example: ALTER TABLE) will open where they were closed (if possible), second and higher child dialogues (example: table advanced properties) will always open on top of its ‘parent’ dialogue. Non-resizable dialogues (such as confirmation boxes) will always open on top of their ‘parent’.
* With multiple SSH-tunnelled connections open stopping and re-executing queries in multiple connections in a fast manner could crash SQLyog.
* If more than one comment occurred before a SELECT statement in the editor, the statement was not identified as a SELECT statement by the Query Profiler and the Query Profiler TAB would not display.
* We did not validate client-side if user checked atoincrement option for a bit column with Create/Alter table dialog.
* If an error occurred while renaming a trigger then trigger was lost as SQLyog was not recreating it back.
* Small GUI fixes.

Miscellaneous:
* The default LIMIT setting for DATA tab has been removed. The setting is not required since we introduced table-level persistence for number of rows displayed. The default for new tables that have not been opened before is 50 – but when user changes the value and next ‘refresh’es SQLyog will save the LIMIT for that particular table persistently across sessions. This in combination with page-wise display in RESULT tab results in a more uniform User Interface for DATA and RESULT tabs.

Downloads: http://webyog.com/en/downloads.php
Purchase: http://webyog.com/en/buy.php

I’ve submitted a Birds of a Feather session for Maatkit at the upcoming MySQL conference. It’s not on the public schedule yet, but it has been accepted and scheduled for 19:00 on 13 Apr 2010. See you there!

Related posts:

1. Presentation uploaded for Maatkit talk at MySQL Conference The slides
2. I’ll be speaking at the O’Reilly MySQL Conference 2010 I’m
3. Learn about Maatkit at the MySQL Conference I’m

Related posts brought to you by Yet Another Related Posts Plugin.

TRUNCATE is usually a fast operation (much faster than DELETE FROM). But sometimes it just hangs; I’ve has several such uncheerful events with InnoDB (Plugin) tables which were extensively written to. The TRUNCATE hanged; nothing else would work; minutes pass.

TRUNCATE on tables with no FOREIGN KEYs should act fast: it translate to dropping the table and creating a new one (and it all depends on the MySQL version, see the manual).

What’s faster than TRUNCATE, then? If you don’t have triggers nor FOREIGN KEYs, a RENAME TABLE can come to the rescue. Instead of:

TRUNCATE log_table

Do:

CREATE TABLE log_table_new LIKE log_table;
RENAME TABLE log_table TO log_table_old, log_table_new TO log_table;
DROP TABLE log_table_old;

I found this to work well for me. Do note that AUTO_INCREMENT values can be tricky here: the “new” table is created with an AUTO_INCREMENT value which is immediately taken in the “working” table. If you care about not using same AUTO_INCREMENT values, you can:

ALTER TABLE log_table_new AUTO_INCREMENT=some high enough value;

Just before renaming.

I do not have a good explanation as for why the RENAME TABLE succeeds to respond faster than TRUNCATE.

I'm happy to announce that my talk "Making MySQL administration a breeze - a look into a MySQL DBA's toolchest" has been accepted for this year's edition of the MySQL Conference & Expo in Santa Clara, which will take place on April 12-15, 2010. The session is currently scheduled for Wednesday 14th, 10:50 in Ballroom E.

My plan is to provide an overview over the most popular utilities and applications that a MySQL DBA should be aware of to make his life easier. The focus will be on Linux/Unix applications available under opensource licenses that ease tasks related to user administration, setting up and administering replication setups, performing backups and security audits.

Of course I will cover the usual suspects (e.g. Maatkit), some of these are actually collections of different utilities by themselves. As it's impossible to go over each individual component in the given time frame, I will try to pick out the most popular/useful parts related to the scopes mentioned above. But I will also cover some lesser known gems that migh be worth taking a look at. What's your the most valued tool in your toolchest? I am still looking for more inspiration.

I look forward to being at the conference again and meeting with colleagues and friends in the MySQL community. Judging from the current schedule, it will be a very interesting mix of talks.

If you're interested in attending, you should consider registering soon! The early registration ends on March 15th. Until then, I encourage you to make use of this "Friend of Speaker" discount code (25% off): mys10fsp

At the 2010 O’Reilly MySQL Conference and Expo there will be a Drizzle BoF!

It’s currently scheduled for 7pm on April 13th.

Come along, it will be awesome.

Dear Lazy Web,

What should the result of the SELECT be below? Assume InnoDB for all storage engines.

CREATE TABLE t1 (a int, b int);
insert into t1 values (1,1),(1,2);
CREATE TEMPORARY TABLE t2 (a int, b int, primary key (a));
BEGIN;
INSERT INTO t2 values(100,100);
CREATE TEMPORARY TABLE IF NOT EXISTS t2 (primary key (a)) select * from t1;

# The above statement will correctly produce an ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
# What should the below result be?

SELECT * from t2;
COMMIT;

PlanetMySQL Voting:
	 Vote UP /
	 Vote DOWN 

# qsh.pl -Mcg all_servers --user root --ask-pass --db=test -e 'SHOW GLOBAL STATUS'  2>error.log
{snip ... lots of output}
Done. Total time 2.919

cat error.log
myserver1: Query Error (1064) You have an error in your SQL syntax ...
myserver2: Query Error (1064) You have an error in your SQL syntax ...

# qsh.pl -Mcm myserver1,myserver2,myserver3 --user root --ask-pass --db=test -e 'SELECT VERSION()'
Password:
myserver1: +-----------------+
myserver1: |       VERSION() |
myserver1: +-----------------+
myserver1: | 4.1.21-standard |
myserver1: +-----------------+
myserver3: +----------------------------+
myserver3: |                  VERSION() |
myserver3: +----------------------------+
myserver3: | 5.0.66a-enterprise-gpl-log |
myserver3: +----------------------------+
myserver2: +-----------------+
myserver2: |       VERSION() |
myserver2: +-----------------+
myserver2: | 4.1.21-standard |
myserver2: +-----------------+
Done. Total time 0.063

If you do not know what International Women’s Day is: http://www.internationalwomensday.com/

Start planning your blog posts for Ada Lovelace day now (March 24th, http://findingada.com/ Ada Lovelace Day is an international day of blogging (videologging, podcasting, comic drawing etc.!) to draw attention to the achievements of women in technology and science.)

To that end, I would like to point out all the women currently in science and tech fields that I admire and think are doing great things. I think it would be great if everyone, male or female, made a list like this:

The women that have taught me science/tech along the way:

High School:
Mary Lou Ciavarra (Physics)
Maria Petretti (Pre-Algebra, and Academic Decathlon)
Reneé Fishman (Biology)
Lisa Acquaire (Economics during Academic Decathlon)

College:
Professor Kalpana White (Biology), and in whose fruit fly lab I worked for 2 semesters.
Professor Eve Marder (Introductory Neuroscience)

Though Brandeis does have female faculty in the Computer Science department, I did not manage to have any classes with female Computer Science faculty members.

My current female DBA co-workers at Pythian: Isabel Pinarci (Oracle), Michelle Gutzait (SQL Server), Catherine Chow (Oracle) and Jasmine Wen (Oracle).

And to folks in the greater MySQL/tech community and tech co-workers past and present, especially those I have been inspired and helped by: Tracy Gangwer, Selena Deckelmann (Postgres), Amy Rich, Anne Cross, and more (If I have forgotten you, I apologize!).

MySQL has a number of global buffers, i.e. your SGA. There are also a number of per session/thread buffers that combined with other memory usage constitutes an unbounded PGA. One of the most common errors in mis-configured MySQL environments is the setting of the 4 primary per session buffers thinking they are global buffers.

Global buffers include:

• key_buffer_size – For MyISAM Indexes (note you can define multiple key_buffer’s The MyISAM Key Cache)
• innodb_buffer_pool_size – For Innodb Table/Indexs
• innodb_additional_mem_pool_size – Innodb additional data dictionary data
• query_cache_size – The MySQL Query Cache

The four important per session buffers are:

• read_buffer_size
• read_rnd_buffer_size
• sort_buffer_size
• join_buffer_size

I have seen people see these values > 5M. The defaults range from 128K to 256K. My advice for any values above 256K is simple. What proof do you have this works better? When nothing is forthcoming, the first move is to revert to defaults or a maximum of 256K for some benchmarkable results. The primary reason for this is MySQL internally as quoted by Monty Taylor – for values > 256K, it uses mmap() instead of malloc() for memory allocation.

These are not all the per session buffers you need to be aware of. Others include thread_stack, max_allowed_packet,binlog_cache_size and most importantly max_connections.

MySQL also uses memory in other areas most noticeably in internal temporary tables and MEMORY based tables.

As I mentioned, there is no bound for the total process memory allocation for MySQL, so some incorrectly configured variables can easily blow your memory usage.

References

• Read Buffer performance hit by Monty Taylor

About “Don’t Assume”

“Don’t Assume” is a series of posts to help the Oracle DBA understand, use and appreciate the subtle differences and unique characteristics of the MySQL RDBMS in comparison to Oracle. These points as essential to operate MySQL effectively in a production environment and avoid any loss of data or availability.

For more posts in this series be sure to follow the mysql4oracledba tag and also watch out for MySQL for Oracle DBA presentations.

The MySQLCamp for the Oracle DBA is a series of educational talks all Oracle DBA resources should attend. Two presentations from this series IGNITION and LIFTOFF will be presented at the MySQL Users Conference 2010 in Santa Clara, April 2010 This series also includes JUMPSTART and VELOCITY. If you would like to here these presentations in your area, please contact me.


PlanetMySQL Voting: Vote UP / Vote DOWN

Today, while on IRC in #pentaho I came across a discussion and a published my.cnf. In this configuration I found some grossly incorrect values for per session buffers (see below).

It doesn’t take a MySQL expert to spot the issues, however there is plenty of bad information available on the Internet and developers not knowing MySQL well can easily be mislead. This has spurred me to create a program to rid the world of bad MySQL configuration. While my task is potential infinite, it will enable me to give back and hopefully do a small amount of good. You never know, saving those CPU cycles may save energy and help the planet.

Stay tuned for more details of my program.

[mysqld]
...
sort_buffer_size = 6144K
myisam_sort_buffer_size = 1G
join_buffer_size = 1G
bulk_insert_buffer_size = 1G
read_buffer_size     = 6144K
read_rnd_buffer_size = 6144K
key_buffer_size		= 1024M
max_allowed_packet	= 32M
thread_stack		= 192K
thread_cache_size       = 256

query_cache_limit	= 512M
query_cache_size        = 512M
...

Since I left Sun back in January, folks have been asking what was next. I’m happy to say that I’m going to continue hacking on open source projects like Drizzle and Gearman, but now at the Rackspace Cloud. Not only will I be there, but I get to continue working closely with a few of the amazing Drizzle hackers who have also joined, including Monty Taylor, Jay Pipes, Stewart Smith, and Lee Bieber.

Why Rackspace Cloud? Late last year I was considering what I wanted to do next with the Oracle acquisition looming near, and this was one of the options that presented itself. Rackspace had been a supporter of Drizzle from early on by offering virtual machines to develop and test on, and when talking to some folks more closely, something really hit home. Rackspace provides first-class service and “fanatical” support – they are not a software company. One might ask why an open source software developer would be interested in a company that doesn’t create software or vice-versa, and the answer is that Rackspace wants to find ways to offer the best possible service now and into the future. What better way than to help develop the next generation of service software and get a jump start into integrating this into their architecture? Both the open source community and Rackspace win.

Another thing I learned while talking with Rackspace is that one of their core principles is transparency. This applies to both customer and employees, and anyone within an open source community can appreciate this. The more I learned about the company and the folks within it, the more impressed I was at the lack of internal barriers or “need-to-know” information. One of Drizzle’s core goals is also transparency, from discussing design decisions on public mailing lists and IRC, to having the entire project management infrastructure hosted out in the open at Launchpad.

What does this mean for the Drizzle project? It means continued support for a number of core developers, more infrastructure for development, and most importantly in my eyes, more context. One of the Drizzle tag-lines is “A Lightweight SQL Database for Cloud and Web,” so what better place to develop a database designed for the cloud than on one of the fastest growing cloud platforms. We’ll get a detailed look at the demands, get feedback from cloud customers, and have the perfect test bed for offering new services. We’ll also be able to work closely with a top-notch group of DBAs, developers, and sysadmins in one of the most demanding service architectures out there. This invaluable context will help the Drizzle developers make more informed decisions moving forward, which also means better software for the community.

Personally, this also means getting back to my hosting roots. Before Sun, I worked at Concentric for almost 10 years in a clustered hosting environment. I’m very familiar with many of the multi-tenant scalability concerns Rackspace has, and I’m excited to be working in this type of environment again. We’ve already been working closely with the MySQL DBAs at Rackspace to learn what the biggest pain points are for a multi-tenant architecture, and we’ll be taking steps to address these as it will help anyone wanting to run Drizzle in a cloud-like environment. Drizzle’s modular architecture has already proved useful, as some of these concerns are easily answered with “oh, we have a plugin point for that.”

I’m excited, this is going to be a fun ride.

Let me start by saying the majority of environments never experience problems of MySQL crashing. I have seen production environments up for years. On my own server I have seen 575 days of MySQL uptime and the problem was hardware, not MySQL.

However it does occur, and the reasons may be obscure.

Confirming mysqld has crashed

To the unsuspecting, MySQL may indeed be crashing and you never know about it. The reason is because most MySQL installations have two running processes, these are mysqld and mysqld_safe.

ps -ef | grep mysqld
root     28822     1  0 Feb22 ?        00:00:00 /bin/sh bin/mysqld_safe
mysql    28910 28822  0 Feb22 ?        00:30:08 /opt/mysql51/bin/mysqld --basedir=/opt/mysql51 --datadir=/opt/mysql51/data --user=mysql --log-error=/opt/mysql51/log/error.log --pid-file=/opt/mysql51/data/dc1.onegreendog.com.pid

One of the functions of mysqld_safe is to restart mysqld if it fails. Unless you review your mysql error log and for low volume systems you will never know. Hint Have you checked your MySQL error log today?

You can determine quickly via SQL your instance uptime.

mysql> SHOW GLOBAL STATUS LIKE '%uptime%';
+---------------+---------+
| Variable_name | Value   |
+---------------+---------+
| Uptime        | 1033722 |
+---------------+---------+
1 row in set (0.00 sec)

This is the number of seconds since start time. While not easily readable for humans, this is more user friendly display. (NOTE: Works for 5.1+ only)

mysql> SELECT FROM_UNIXTIME(UNIX_TIMESTAMP() - variable_value) AS server_start
FROM INFORMATION_SCHEMA.GLOBAL_STATUS
WHERE variable_name='Uptime';
+---------------------+
| server_start        |
+---------------------+
| 2010-02-22 15:22:13 |
+---------------------+
1 row in set (0.07 sec)

Debugging a mysqld core file

When correctly configured, mysqld will generate a core file (See How to crash mysqld intentionally for background information on required settings).

Your first check is to determine if the mysqld binary used has debugging information and symbols stripped. You need this information not stripped for identifying symbol names.

$ file bin/mysqld
bin/mysqld: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0,
dynamically linked (uses shared libs), for GNU/Linux 2.4.0, not stripped

You can use gdb and with a backtrace command (bt) you can see a stack trace of calls. This won’t help the average DBA without C or MySQL internal knowledge greatly, however it’s essential information to get to the bottom of the problem.

In the following example I’m going to use Bug #38508 to intentionally crash my test instance.

mysql> drop table if exists t1,t2;
mysql> create table t1(a bigint);
mysql> create table t2(b tinyint);
mysql> insert into t2 values (null);
mysql> prepare stmt from "select 1 from t1 join  t2 on a xor b where b > 1  and a =1";
mysql> execute stmt;
mysql> execute stmt;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Lost connection is the first sign of a problem. We check the error log to confirm.

$ tail data/`hostname`.err
100306 14:51:49 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338301 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x521f160
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x401b6100 thread_stack 0x40000
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(my_print_stacktrace+0x2e)[0x8abfbe]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(handle_segfault+0x322)[0x5df252]
/lib64/libpthread.so.0[0x35fb00de80]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_ZN9Item_cond10fix_fieldsEP3THDPP4Item+0x7f)[0x5654ff]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_ZN9Item_cond10fix_fieldsEP3THDPP4Item+0xb8)[0x565538]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z11setup_condsP3THDP10TABLE_LISTS2_PP4Item+0xf6)[0x621f96]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_ZN4JOIN7prepareEPPP4ItemP10TABLE_LISTjS1_jP8st_orderS7_S1_S7_P13st_select_lexP18st_select_lex_unit+0x2db)[0x645f3b]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x7a4)[0x654d24]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x16c)[0x659f9c]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld[0x5ec92a]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z21mysql_execute_commandP3THD+0x602)[0x5efb22]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_ZN18Prepared_statement7executeEP6Stringb+0x3bd)[0x66587d]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x7c)[0x66874c]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z22mysql_sql_stmt_executeP3THD+0xa7)[0x668c27]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1123)[0x5f0643]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x357)[0x5f5047]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xe93)[0x5f5ee3]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(_Z10do_commandP3THD+0xe6)[0x5f67a6]
/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld(handle_one_connection+0x246)[0x5e9146]
/lib64/libpthread.so.0[0x35fb006307]
/lib64/libc.so.6(clone+0x6d)[0x35fa4d1ded]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x5249320 = select 1 from t1 join  t2 on a xor b where b > 1  and a =1
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
100306 14:51:49 mysqld_safe Number of processes running now: 0
100306 14:51:49 mysqld_safe mysqld restarted
100306 14:51:49 [Note] Plugin 'FEDERATED' is disabled.
100306 14:51:50  InnoDB: Started; log sequence number 0 44233
100306 14:51:50 [Note] Event Scheduler: Loaded 0 events
100306 14:51:50 [Note] /home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld: ready for connections.
Version: '5.1.38'  socket: '/tmp/mysql.sock.3999'  port: 3999  MySQL Community Server (GPL)

Confirming we got the “Writing a core file” line, we can find and use this.

$ find . -name "core*"
./data/core.23290

$ gdb bin/mysqld data/core.23290
GNU gdb Red Hat Linux (6.5-37.el5_2.2rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db library "/lib64/libthread_db.so.1".

Reading symbols from /lib64/libpthread.so.0...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libnsl.so.1...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /lib64/libm.so.6...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libgcc_s.so.1...done.
Loaded symbols for /lib64/libgcc_s.so.1
Core was generated by `/home/rbradfor/mysql/mysql-5.1.38-linux-x86_64-glibc23/bin/mysqld --defaults-fi'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000035fb00b142 in pthread_kill () from /lib64/libpthread.so.0
(gdb) bt
#0  0x00000035fb00b142 in pthread_kill () from /lib64/libpthread.so.0
#1  0x00000000005df285 in handle_segfault (sig=11) at mysqld.cc:2552
#2  
#3  0x00000000005654ff in Item_cond::fix_fields (this=0x5249dd0, thd=0x521f160, ref=) at item_cmpfunc.cc:3900
#4  0x0000000000565538 in Item_cond::fix_fields (this=0x52435b8, thd=0x521f160, ref=) at item_cmpfunc.cc:3912
#5  0x0000000000621f96 in setup_conds (thd=0x521f160, tables=, leaves=0x52494d0, conds=0x5244e38) at sql_base.cc:7988
#6  0x0000000000645f3b in JOIN::prepare (this=0x5243770, rref_pointer_array=0x5248a90, tables_init=, wild_num=, conds_init=,
    og_num=, order_init=0x0, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x52488c0, unit_arg=0x5248498) at sql_select.cc:412
#7  0x0000000000654d24 in mysql_select (thd=0x521f160, rref_pointer_array=0x5c3fd0, tables=0x4, wild_num=0, fields=@0x52489c8, conds=0x52435b8, og_num=0, order=0x0, group=0x0, having=0x0,
    proc_param=0x0, select_options=0, result=0x52688a0, unit=0x5248498, select_lex=0x52488c0) at sql_select.cc:2377
#8  0x0000000000659f9c in handle_select (thd=0x521f160, lex=0x52483f8, result=0x52688a0, setup_tables_done_option=0) at sql_select.cc:268
#9  0x00000000005ec92a in execute_sqlcom_select (thd=0x521f160, all_tables=0x52494d0) at sql_parse.cc:5011
#10 0x00000000005efb22 in mysql_execute_command (thd=0x521f160) at sql_parse.cc:2206
#11 0x000000000066587d in Prepared_statement::execute (this=0x5245d60, expanded_query=, open_cursor=false) at sql_prepare.cc:3579
#12 0x000000000066874c in Prepared_statement::execute_loop (this=0x5245d60, expanded_query=0x401b43c0, open_cursor=false, packet=, packet_end=)
    at sql_prepare.cc:3253
#13 0x0000000000668c27 in mysql_sql_stmt_execute (thd=) at sql_prepare.cc:2524
#14 0x00000000005f0643 in mysql_execute_command (thd=0x521f160) at sql_parse.cc:2215
#15 0x00000000005f5047 in mysql_parse (thd=0x521f160, inBuf=0x5243520 "execute stmt", length=12, found_semicolon=0x401b6060) at sql_parse.cc:5931
#16 0x00000000005f5ee3 in dispatch_command (command=COM_QUERY, thd=0x521f160, packet=0x525fde1 "execute stmt", packet_length=) at sql_parse.cc:1213
#17 0x00000000005f67a6 in do_command (thd=0x521f160) at sql_parse.cc:854
#18 0x00000000005e9146 in handle_one_connection (arg=dwarf2_read_address: Corrupted DWARF expression.
) at sql_connect.cc:1127
#19 0x00000035fb006307 in start_thread () from /lib64/libpthread.so.0
#20 0x00000035fa4d1ded in clone () from /lib64/libc.so.6
(gdb) quit

You can use gdb to obtain additional information based on the type of information available.

Now what?

Is the problem a bug? Is it data corruption? Is it hardware related?

Gathering the information is the first step in informing you of more detail that will enable you to search, discuss and seek professional advice to address your problem.

References

• MySQL Internals – Debugging a MySQL Server
• What to Do If MySQL Keeps Crashing
• Debugging mysqld under gdb
• Hunting the core – An old by good intro article to cores and MySQL

Although a few folks knew about where I and many of the Sun Drizzle team had ended up, we've waited until today to "officially" tell folks what's up. We — Monty Taylor, Eric Day, Stewart Smith, Lee Bieber, and myself — are all now "Rackers", working at Rackspace Cloud. And yep, we're still workin' on Drizzle. That's the short story. Read on for the longer one

An Interesting Almost 3 Years at MySQL

I left my previous position of Community Relations Manager at MySQL to begin working on Brian Aker's newfangled Drizzle project in October 2008.

Many people at MySQL still think that I abandoned MySQL when I did so. I did not. I merely had gotten frustrated with the slow pace of change in the MySQL engineering department and its resistance to transparency. Sure, over the 3 years I was at MySQL, the engineering department opened up a bit, but it was far from the ideal level of transparency I had hoped to inspire when I joined MySQL.

For almost 3 years, I had sent numerous emails to the MySQL internal email discussion lists asking the engineering and marketing departments, both headed by Zack Urlocker, to recognize the importance and necessity of major refactoring of the MySQL kernel, and the need to modularize the kernel or risk having more modular databases overtake MySQL as the key web infrastructure database. The focus was always on the short term; on keeping up with the Jones' as far as features went, and I railed against this kind of roadmap, instead pushing the idea of breaking up the server into modules that could be blackboxed and developed independently of the kernel. My ideas were met with mostly kind responses, but nothing ever materialized as far as major refactoring efforts were concerned.

I remember Jim Winstead casually responding to one of my emails, "Congratulations, you've just reinvented Apache 2.0". And, yes, Jim, that was kind of the point...

The MySQL source code base had gotten increasingly unmaintainable over the years, and key engineers were extremely resistant to changing the internals of MySQL and modernizing it. There were some good reasons for being resistant, and some poor reasons (such as "this is the way we've always done it"). Overall, it's tough to question the strategy that Zack, Marten Mickos, and others had regarding the short term gains. After all, they managed to maneuver MySQL into a winning position that Sun Microsystems thought was worth one billion dollars. Because of this, it's tough to argue with them.

Working on Drizzle since October 2008 (officially)

I'm not the kind of person which likes to wait for years to see change, and so the Drizzle project interested me because it was not concerned with backwards compatibility with MySQL, it wasn't concerned with having a roadmap that was dependent on the whims of a few big customers, and it was very much interested in challenging the assumptions built into a 20 year-old code base. This is a project I could sink my teeth into. And I did.

Many folks have said that the only reason Drizzle is still around is because Sun continued to pay for a number of engineers to work on Drizzle as "an experiment of sorts" and that Drizzle has no customers and therefore nothing to lose and everything to gain. This was true, no doubt about it. At Sun CTO Labs, the few of us did have the ability to code on Drizzle without the pressure-cooker of product marketing and sales demands. We were lucky.

4 6 9 10 Months in Purgatory

So, around rolls April 2009. The stock market and worldwide economy had collapsed and recession was in the air. There's one thing that is absolutely certain in recession economies: companies that have poor leadership and direction and are beholden to the interests of a large stockholder will seek an end to their misery through acquisition by a larger, stronger firm.

And Sun Microsystems was no different. JAVA stock plummeted to two dollars a share, and Jonathan Schwartz and the Sun board began shopping Sun around to the highest bidder. IBM was courted along with other tech giants. So was Oracle.

And it was with a bit of a hangover that I awoke at the MySQL conference in April 2009 to the news that Oracle had purchased Sun Microsystems. Joy. We'd just gone through 14 months of ongoing integration with Sun Microsystems and now it was going to start all over again.

Anyone who follows PlanetMySQL knows about the ensuing battle in the European Commission's court regarding monopoly of Oracle in the database market with its acquisition of MySQL. Monty Widenius, Eben Moglen, even Richard Stallman, weighed in on the pros and cons of Oracle's impending control over MySQL.

All the while, us Sun Microsystems employees had to hold our tongues and try to keep our jobs as Sun laid off thousands more workers while the EC battle ensued. Not fun. It was the employment equivalent of purgatory. And the time just dragged on, with many employees, including myself and the Sun Drizzle team, not having a clue as to what would happen to us. Management was completely silent about future plans. Oracle made zero attempts to outline its future strategy regarding software, and thus most software employees simply kept on doing their work not knowing if the pink slip was arriving tomorrow or not. Lots of fun that was.

Oracle Doesn't Need Our Services — Larry Don't Need No Stinkin' Cloud

The acquisition finally closed and very shortly afterwards, I got a call from my boss, Lee Bieber, that Oracle wouldn't be needing our services. Monty, Eric, and Stewart had already resigned; none of them had any desire to work for Oracle. Lee and I had decided to see what they had in mind for us. Apparently, not much.

Larry Ellison has gone on record that the whole "cloud thing" is faddish. I don't know whether Larry understands that cloud computing and infrastructure-as-a-service, platform-as-a-service, and database-as-a-service will eventually put his beloved Oracle cash cow in its place or not. I don't know whether Oracle is planning on embracing the cloud environments which will continue to eat up the market share of more traditional in-house environments upon which their revenue streams depend. I really don't.

But what I do know is that Rackspace is betting that providing these services is what the future of technology will be about.

Happiness is a Warm Cloud

Our team has landed at Rackspace Cloud. I've now been down to San Antonio twice to meet with key individuals with whom we'll be working closely. Rackspace is not shy about why the wanted to acquire our team. They see Drizzle as a database that will provide them an infrastructure piece that will be modular and scalable enough to meet the needs of their very diverse Cloud customers, of which there are many tens of thousands.

Rackspace recognizes that the pain points they feel with traditional MySQL cannot be solved with simple hacks and workarounds, and that to service the needs of so many customers, they will need a database server that thinks of itself as a friendly piece of their infrastructure and not the driver of its applications. Drizzle's core principles of flexibility and focus on scalability align with the goals Rackspace Cloud has for its platform's future.

Rackspace is also heavily invested in Cassandra, and sees integration of Drizzle and Cassandra as being a key way to add value to its platforms and therefore for its customers.

Rackspace is all about the customers, and this is a really cool thing to experience. It's typical for companies to claim they are all about the customer — in fact, every company I've ever worked for has claimed this. Rackspace is the first company I've worked for where you actually feel this spirit, though. You can see the fanaticism of Rackers and how they view what they do always in terms of service to the customer. It's infectious, and I'm pretty psyched to be on their team.

Anyway, that's my story and I'm stickin' to it. See y'all on the nets.

It is in

As of 2010-03-06, the performance schema is merged into MySQL version 5.5.3-m3, in mysql-trunk.

The documentation for 5.5 has also been updated, and contains a new MySQL Performance Schema chapter.

Disclaimer

Please note that MySQL 5.5 is not a GA product. The performance schema feature may still change at any time, for any reason, and without notice. Customers should not make purchasing decisions based on the availability of the performance schema.

It seems that a lot of people equate non-SQL databases with non-relational-ness, or malign the word relational. This is pretty much pure ignorance. If you’ve ever uttered a sentence that includes the phrase “…non-relational database…” then I have two suggestions for you.

1. Study relational algebra. At a bare minimum, read the Wikipedia article on relational algebra. There is much more you could do — take a class on the topic, or read C.J. Date’s SQL and Relational Theory (my review). Ask yourself how similar SQL is to the relational algebra. How is relational algebra different from SELECT and GROUP BY? Is relational theory about relationships between data? What part do transactions play in relational algebra? Is MySQL a relational database? What about PostgreSQL, Oracle, or DB2?
2. Now that you understand relational theory more, choose a database that you think is non-relational and write a formal proof that it is not relationally complete. Please do post a link to the proof in the comments.

The truth is, a non-relational database would be of very little use. In layman’s terms, it would mean you have some data that represents true statements, and a piece of software designed to answer questions using those facts, and you can’t answer simple first-order logic questions with the software. How is this an improvement? How is this useful?

Related posts:

1. A review of SQL and Relational Theory by C. J. Date SQL and Re
2. InnoDB is a NoSQL database As long as
3. On the unhelpfulness of NoSQL My favorit

Related posts brought to you by Yet Another Related Posts Plugin.

MySQL, the open source database product that puts the "M" in LAMP, was created by MySQL AB, a company founded in 1995 in Sweden. In 2008, MySQL AB announced that it had agreed to be acquired by Sun Microsystems for approximately $1 billion.

The story of MySQL AB is pretty amazing, so I unleashed my "inner academic", did some research and compiled a timeline of MySQL AB's history. This timeline is assembled based on different resources online, such as MySQL press releases (example 1) and interviews with MySQL AB executives (example 2, example 3), etc.

Things to add? Let me know in the comments and I'll update the post.

1995

• MySQL AB founded by Michael Widenius (Monty), David Axmark and Allan Larsson in Sweden.

2000

• MySQL goes Open Source and releases software under the terms of the GPL. Revenues dropped 80% as a result, and it took a year to make up for it.

2001

• Mårten Mickos elected CEO at age 38. Mårten was the CEO of a number of Nordic companies before joining MySQL, and comes with a sales and marketing background.
• 2 million active installations.
• Raised series A with undisclosed amount from Scandinavian venture capitalists. Estimated to be around $1 to $2 million.

2002

• MySQL launched US headquarters in addition to Swedish headquarters.
• 3 million active users.
• Ended the year with $6.5 million in revenue with 1,000 paying customers.

2003

• Raised a $19.5 million series B from Benchmark Capital and Index Ventures.
• 4 million active installations and over 30,000 downloads per day.
• Ended the year with $12 million in revenue.

2004

• With the main revenue coming from the OEM dual-licensing model, MySQL decides to move more into the enterprise market and to focus more on recurring revenue from end users rather than one-time licensing fees from their OEM partners.
• Ended the year with $20 million in revenue.

2005

• MySQL launched the MySQL Network modeled after the RedHat Network. The MySQL Network is a subscription service targeted at end users that provides updates, alerts, notifications, and product-level support designed to make it easier for companies to manage hundreds of MySQL servers.
• MySQL 5 ships and includes many new features to go after enterprise users (e.g. stored procedures, triggers, views, cursors, distributed transactions, federated storage engines, etc.)
• Oracle buys Innobase, the 4-person Finnish company behind MySQL's InnoDB storage backend.
• Ended the year with $34 million in revenue based on 3400 customers.

2006

• Mårten Mickos confirms that Oracle tried to buy MySQL. Oracle' CEO Larry Ellison commented: "We've spoken to them, in fact we've spoken to almost everyone. Are we interested? It's a tiny company. I think the revenues from MySQL are between $30 million and $40 million. Oracle's revenue next year is $15 billion."
• Oracle buys Sleepycat, the company that provides MySQL with the Berkeley DB transactional storage engine.
• Mårten Mickos announces that they are making MySQL ready for an IPO in 2008 on an projected $100 million in revenues.
• 8 million active installations.
• MySQL has 320 employees in 25 countries, 70 percent of whom work from home.
• Raised a $18 million Series C based on a rumored valuation north of $300 million.
• MySQL is estimated to have a 33% market share measured in install base and 0.2% market share measured in revenue (the database market was a $15 billion market in 2006).
• Ended the year with $50 million in revenue.

2007

• Ended the year with $75 million in revenue.

2008

• Sun Microsystems acquired MySQL AB for approximately $1 billion.
• Michael Widenius (Monty) and David Axmark, two of MySQL AB's co-founders, begin to criticize Sun publicly and leave Sun shortly after.

2009

• Mårten Mickos leaves Sun and becomes entrepreneur-in-residence at Benchmark Capital. Sun has now lost the business and spiritual leaders that turned MySQL into a success.
• Sun Microsystems and Oracle announced that they have entered into a definitive agreement under which Oracle will acquire Sun common stock for $9.50 per share in cash. The transaction is valued at approximately $7.4 billion.

During my previous post on how to improve derived tables performance, I patched the code to add indexes forcefully on internal derived table results, which made a huge difference in the performance. It was just an experiment and a thought to see if it really works without re-writing the queries, so that the logic can be pushed towards the engine rather than query re-write. \

But I got few emails in my inbox today asking whether MySQL really create any keys on internal temporary tables.

The answer is YES; and MySQL does create two keys on internal temporary tables namely ‘group_key‘ and ‘distinct_key‘ on the following conditions:

• If there is any aggregate function and/or group-by (group_key)
• Distinct column name(group_key)
• Distinct in combination with group-by/aggregation functions (distinct_key)

Provided the query results are yielded in temporary table (Using temporary from the explain), else they get optimized away by the existing indexes from the regular table itself. These keys are added to both memory and disk based (MyISAM) internal temporary tables; so it does not matter if the internal temporary table is in memory or disk.

Here is a simple dump of internal temporary table index stats for some of the basic queries related to Information schema [Warning: these queries are really bad, and can't be used for any production use as they are meant for demonstration of different internal keys ]. This is a patch that I might be using for SHOW TEMPORARY TABLES when internal tables are included in the second version. The first version of the patch is already pushed to Maria branch, hoping that it gets pushed to 5.1.

-----------------------------
 TMP TABLE STATS, SESSION: 1
   temp file  : /tmp/#sqlf90_1_1f
   temp type  : MEMORY
   index count: 1
    key 1-1   : distinct_key
    field     : (null)
    key 1-2   : distinct_key
    field     : ENGINE
 query: select count(distinct engine) from information_schema.tables
-----------------------------
 
-----------------------------
 TMP TABLE STATS, SESSION: 1
   temp file  : /tmp/#sqlf90_1_21
   temp type  : MEMORY
   index count: 1
    key 1-1   : group_key
    field     : TABLE_NAME
 query: select table_name, sum(data_length+index_length) from information_schema.tables 
        where table_schema='mysql' group by 1
-----------------------------
 
-----------------------------
 TMP TABLE STATS, SESSION: 1
   temp file  : /tmp/#sqlf90_1_24
   temp type  : MEMORY
   index count: 1
    key 1-1   : group_key
    field     : TABLE_SCHEMA
    key 1-2   : group_key
    field     : TABLE_NAME
    key 1-3   : group_key
    field     : COLUMN_NAME
 query: select tab.table_schema, tab.table_name, column_name, index_name, seq_in_index 
        from Information_schema.tables tab join information_schema.statistics stast 
        using(table_schema,table_name) group by  1,2,3
-----------------------------
 
-----------------------------
 TMP TABLE STATS, SESSION: 1
   temp file  : /tmp/#sqlf90_1_bd
   temp type  : MEMORY
   index count: 1
    key 1-1   : group_key
    field     : TABLE_NAME
    key 1-2   : group_key
    field     : TABLE_TYPE
    key 1-3   : group_key
    field     : ENGINE
    key 1-4   : group_key
    field     : INDEX_SCHEMA
    key 1-5   : group_key
    field     : INDEX_NAME
 query: select tab.table_schema,  tab.table_name, table_type, engine, index_schema, 
        index_name from information_schema.tables tab join information_schema.statistics
        stats using(table_schema, table_name) where table_schema='mysql' group by 
        1,2,3,4,5, 6 order by 4,3,2,1
-----------------------------
 
-----------------------------
 TMP TABLE STATS, SESSION: 1
   temp file  : /tmp/#sqlf90_1_e0
   temp type  : MEMORY
   index count: 1
    key 1-1   : group_key
    field     : TABLE_NAME
 query: select  table_name, sum(data_length+index_length) from information_schema.tables
        where table_schema='mysql' group by 1
-----------------------------

DESCRIPTION
maxfiles increases the number of files that a volume can
hold, as close as possible to max. File inodes are stored
in blocks, and the filer may round the requested max num-
ber of files to the nearest block.

Once increased, the value of max can never be lowered, so
the new value must be larger than the current value.