Favorite Quotes: Just A Mess Without A Clue
In Data and Technology Today, Craig Mullins shared his favorite quotes that apply to database professionals. Here is one of my own favorite quotes. In The Tao of Pooh by Benjamin Hoff , Winnie the Pooh—the “tubby little cubby all stuffed with fluff”—says:
“And if you don’t know Which to Do
Of all the things in front of you,
Then what you’ll have when you are through
Is just a mess without a clue …”
What are the deliverables of the database administration role? How likely is it that the deliverables will be completed if you cannot articulate the deliverables? “Deliverables” are not the same as “current priorities,” because priorities change from day to day; performance improvement may be your priority today but it may not be your priority when performance returns to acceptable levels. Deliverables are not the same as “assigned tasks,” either, because assigned tasks change from day to day; resetting passwords for forgetful users may be one of your assigned tasks today but you may not have do it any more if, for example, your organization begins using self-service technology or single sign-on technology, or if the task is assigned to a Service Desk. An example of a deliverable is “databases that meet the needs of the business”—the deliverable does not change from day to day. If there is only one database administrator in the organization, then it is the individual’s deliverable. If database administration is performed by a team, it is a shared deliverable.
How I Became a DBA
I became an Oracle DBA by accident. I was supporting another database technology when Big Bob, the Oracle DBA at my then employer, suddenly resigned. I was asked to take over because I had expressed an interest in becoming an Oracle DBA.
I created a documentation template and asked Bob to spend his remaining time documenting each database using the template. He protested that he did not have the time to provide so much detail and suggested that we meet for an hour or two.
At the meeting, Bob told me not to worry and that I was a smart kid and would soon learn my way around. We ran through the list of databases in about half an hour, spending less than a minute on each while I hastily scribbled notes.
But it wasn’t Big Bob’s fault at all, because documentation and record-keeping were not organizational priorities. When the time came for me to take over from him, Big Bob had little more for me than a few passwords, a firm handshake, and lots of good wishes.
Start with the End in Mind: The Ten Deliverables
In his best-selling book The 7 Habits of Highly Effective People (Free Press, 1989), Stephen Covey distills the secrets of effectiveness into seven principles. In my opinion, the most important habit is “Start with the End in Mind”—how you want your work to be evaluated when you have completed it. When Big Bob turned responsibilities over to me, he had little more for me than the database passwords, a firm handshake, and good wishes—he left me very unhappy.
To be effective as database administrators, we must start with the end in mind—the moment when we hand over responsibilities to our successors. What will we give them other than the database passwords, a firm handshake, and good wishes?
Here are the ten deliverables of the database administration role—they map to the ten deliverables of the Operations team which are listed in the ITIL literature. In large organizations with many database administrators, these are shared deliverables.
A database that meets the needs of the business: This deliverable is the most important deliverable, if not the only one. The needs of the business include certain levels of performance, security, and availability. You must understand the needs of the business, you must have a way of evaluating how well the needs are met, and you must have a methodology for meeting those needs. Any chronic performance, security, and availability issues must be discussed with the incoming database administrator.
A secure document library: The absence of a document repository causes a lot of valuable information to be lost. Examples of documents that should be retained include service level agreements, network diagrams, architecture diagrams, licensing information, E-R diagrams, performance reports, audit reports, software manuals, installation notes, project notes, copies of important correspondence, and so on. “Standard Operating Procedures” (SOPs) are another important class of documents. Original software media and files should also be stored in the library for use if the database needs to be rebuilt or if additional databases need to be created. Note that the document library needs to be secure since it contains sensitive and confidential information.
Work logs of service requests, alarms, and changes: Work logs are important for many reasons. They bring transparency and visibility to the database administration function. From the Incident Management perspective, it is necessary to review the work logs and identify inefficiencies and root causes. From the Problem Management perspective, it is necessary to review the work logs and identify chronic problems. From the Availability Management perspective, it is necessary to review the work logs and identify availability issues. These are just some examples of how work logs help bring about improvements and efficiencies in your ability to provide good service to the business.
Standard Operating Procedures: Any database administration task that is done repeatedly should be codified into a Standard Operating Procedure (SOP). Using a written SOP helps efficiency and accuracy.
Procedures and records for backup testing and failover testing: It is absolutely essential that backup procedures and disaster recovery procedures be documented. The procedures should be periodically tested and records should be maintained.
Maintenance and batch schedules, documentation, and records: Database maintenance procedures should be documented and records should be maintained. If the maintenance procedures are automated, log records should also be automatically created. For example, an RMAN catalog can be used to store backup histories. Any repeating tasks or batch jobs that are the responsibility of the database administrator should also be adequately documented and have records maintained for them.
Database administration tools: Database administration tools include Oracle-supplied tools such as Database Control, Grid Control, and SQL Developer. The Management Packs, such as Diagnostics Pack, Tuning Pack, Change Management Pack, and Configuration Pack, are very valuable tools but most organizations don’t purchase licenses to use them because of their high cost and because Enterprise Edition is a prerequisite. Other popular tools are Toad from Quest Software and DBArtisan from Embarcadero Technologies.
Management reports: Examples of database reports for management are reports on database growth, workload, and performance. STATSPACK and AWR histories should be retained for as long as practicable. I suggest retaining data for at least one year if you can afford the space. Baseline snapshots should be retained indefinitely. For example, you can designate the period between 9 a.m. and 10 a.m. every Monday morning as a baseline period so that the snapshots marking the beginning and end of the period are retained indefinitely.
Exception reports: This deliverable includes reports on SLA violations, security violations, backup failures, and the like. For example, a certain stored procedure or SQL statement may have been identified as critical to the business, and an exception report can be produced by mining STATSPACK data.
Audit reports: This deliverable typically refers to audit reports conducted by security auditors but can also refer to internal audits of compliance with organizational processes such as Change Management or database reviews by external consultants. The absence of audits indicates a lack of oversight of the database management function.
The Book You Really Need
Do you know how to start or stop a database? I thought I did—until I went to work in a large Network Operations Center. We had Solaris, AIX, HP/UX, Linux, and Windows. We had Oracle 8i, Oracle 9i, and Oracle 10g. We had VCS, HP Service Guard, Sun Clusters, Data Guard, and RAC. There were so many variations of the startup and shutdown procedures that I could not remember all of them.
The book that you really need will never be found in book stores—it is the book containing all the procedures that you need to operate your databases. You’re going to have to write that book yourself. Nobody can write the book for you, because you have a unique environment and nobody except you would write a book that caters to a unique environment.