Adroit Project & Team Management Spurs Notable Success
Company was starting a new program to release an upgraded Java Virtual Machine software product on its operating platform. Project would take 20 people 2.5 years to develop and release. Port Java “HotSpot” technology to MIP processor had never been done before by anyone in the industry. Executive Management wanted program “predictability”: set a date and deliver.
Team lacked good estimating and project management skills, and the project was technically complex. The technology was owned and licensed from a “competitor-partner” who charged high fees for consultations, and the program was on a tight budget. However, expertise was spread across the team and the competitor-partner.
As Program Manager, planned, developed, and directed implementation (with follow-on oversight) of a number of functions/steps to improve team performance: Wideband Delphi sizing; sanity check against COCOMO and similar jobs by team members’ associates in other companies; bottoms-up planning and estimating; earned-value tracking using objective evidence of progress; and rigorous risk management planning and tracking using MS Access-based tool. Identified and took action at early indications of any problems.
Program completed and product shipped successfully within 10% of target date and cost planned 2.5 years earlier, representing a 200% improvement in cost and schedule control over previous middleware software product programs. Product achieved goals of doubling performance over previous Java offering.
Though the team received a little grief for being 10% late, no program in this organization had ever achieved this degree of predictability. In retrospect, a schedule buffer would have ensured a committed plan date and kept reviews more in perspective and positive. Risk management was stellar and averted what could have become disasters by rigorously following through on risk mitigation plans.
Resource & Risk Management Prompt Highly Successful Release
Two major technology companies had just merged. One company’s key offering on existing NonStop Servers was Java Virtual Machine. Servers ran on MIP technology from Silicon Graphics (SGI). Company was developing a new NonStop system using Intel’s new Itanium chip technology, and was also in the process of bringing Sun Microsystem’s latest “HotSpot” compiler technology into its MIP-based Java offering. A similar Java offering was needed for the Itanium system shortly thereafter.
Effort needed to create either of the offerings consumed all available resources, yet both were necessary. Executive management was concerned about the risks involved, as the organization doing the work had no success at predicting schedules and costs.
Post-merger, the team discovered that the new partner company had a HotSpot Java running on another machine using Itanium chip technology but a different operating system. Developed an approach to merge the two programs: HotSpot technology would be ported to NonStop Operating System on the MIP-based product first, and ship before the new Itanium-based system was ready to integrate; then this would be integrated with the partner company’s compiler developed for Itanium for the new server. Used historical cost and schedule to determine most likely performance of the same team doing similar work (this team had ported Java technology to NonStop products three times). Worked with development manager using “Risk Radar” (MS Access-based tool to build risk management plans) to lower risk to acceptable levels and reinforce confidence in the dates. Built plans and briefed management.
Trimmed Itanium program cost to $2.4M (estimated cost=$3.6M). Itanium processor program pressured several times to shorten the date for Itanium Java release. Reviewed basis of estimate and stood by schedules. In the end, Itanium processor program took longer than Java release, which hit exactly on forecast cost and completion date. New NonStop Itanium Java Virtual Machine realized improvements in application processing speed 50% higher than any other NonStop software moving from MIP to Itanium. Team was recognized with many honors and awards, and was not assigned to workforce reduction program.
Resourceful Program Planning Wins Lucrative USAF Contract
As Chief Systems Engineer of science and technology defense contractor, assigned to bid, win and operate USAF contract. Program was two-phased approach for $500M over ten years to operate a global network of real-time flight simulators operating in joint computer-simulated combat exercises. Phase 1 bid required a detailed comprehensive plan with basis of estimates, Work Breakdown Structure (WBS), critical path analysis, cost and schedule detail.
The assigned program manager was also the acquisition manager, was consumed with proposal management workload, and had no idea how to build this plan. Team required expertise from multiple geographically distributed, internal and external organizations. Most of the internal organizations had critical technical expertise and would perform key work under the contract, but did not have the skill sets for this level of detailed planning. While no expert in many of the key areas, possessed best knowledge of how the whole system would integrate and operate.
Consulted with a simulation prime contractor to get a historical perspective of what the customer expected to see in a WBS. Discussed and diagramed technical lifecycles of each key deliverable with appropriate teams. Many artifacts from these discussions were used in technical parts of the proposal. Constructed a WBS for the program organized by milestones, each composed of accomplishments, which were comprised of objective criteria achieved by activities involving the technical lifecycles for each deliverable. Obtained historical basis of deliverables estimates from each team, and determined method to size each deliverable relative to the basis of estimate. Determined activities and duration required for each deliverable from these sizings, using the technical lifecycle for each. Mapped results to the Milestones-Accomplishment-Criteria WBS model, using customer-significant milestones for the top level of the WBS. Validated and finalized with each team involved.
Plan was evaluated by a 14-man team of experts from the government, including key members for the Software Engineering Institute. It was accepted with no questions and the team won Phase 1 of the contract for $2.5M feasibility study for one year. This was critical to getting to the final $500M contract.
Multiple Concurrent Project Management Doubles Profitability
Assumed role as Program Manager of USAF Europe SETA Program for company, responsible for oversight, client satisfaction, contract performance, and profitability of all work performed under this contract. Each task order had a single technical lead on the government side responsible for contract performance for payment, and a single internal technical lead responsible for project management and technical performance. Needed an efficient way to stay on top of all delivery orders, and report predictable and favorable results to upper executive management.
Task orders were specific Firm Fixed Price delivery orders. Company was paid only when contract deliverables arrived on time with acceptable quality, and the company incurred costs for work whether ultimately paid or not. Profit margins bid to the Air Force for the overall contract was less that 10-12%. Technical teams and government contracts were local to Ramstein, Germany area, but business management was stateside. As many as 12 independent task orders ran simultaneously, and deliverables were due weekly throughout the life of the contract.
Team technical leads were not usually skilled project managers. Recruited stateside business manager skilled in earned value and project management, knowledgeable in contract requirements, and skilled with company financial systems. Gave business manager responsibility for working with technical leads to provide valid financial status and metrics. Made technical leads accountable for timeliness and accuracy of reports. Reviewed in person with technical leads and business manager monthly, more often if increased risk warranted.
Required monthly status reporting and implemented single-sheet reporting format that included: summary cost/schedule for each task order; summary task order scoped statement including required deliverables; multiple independent summary cost and schedule forecasts with Earned Value based upon deliverables and intermediate steps to completion, work completed, work remaining, cost performance index, and schedule performance index; actual cost burn rate; critical path analysis; and project schedule and critical paths for each deliverable.
Doubled profitability of contracts and overall division performance. Very satisfied Air Force customers led to follow-on five-year contract and continued new business worth over $25M. Increased number of task orders for same overhead, thereby doubling contract revenue with no increase in contract overhead. Having the right comprehensively skilled business manager to track multiple concurrent projects proved highly successful.
Complex Simulator Design Triggers Multiple Contracts
Company had a contract to develop a cockpit-procedure trainer for the B-1B. The Air Force had new aircraft rolling out before their simulators to train pilots would be ready. They had a smaller amount of money set aside for a limited, interim trainer to get otherwise-experienced pilots initial training on the new aircraft, sufficient to get them in the air to train further. A working flight simulator needed to be designed and built in 18 months.
Neither the government nor the company had flight-simulation capability. Newly formed internal organization in Dayton, OH took on the task, one of their biggest jobs to date. Because of the budget, team had to use all digital computing (an emerging technology), which require sophisticated computing techniques to pull off good flight dynamics.
Secured wind tunnel and flight test data showing the stability and control derivative for the B-1B. Obtained mass and inertial data, fuel loadings, and similar aircraft design data. Requested flight-control block diagrams for the aircraft. Recruited an Air Force staffer with flight simulator background, and hired one new computer science graduate and one senior engineer who knew aeronautics, but not simulators. Used structure design techniques to rapidly develop entire flight station model that would process signals from the flight controls, compute resulting dynamics, and output the required aircraft states to the instrumentation. Calculated necessary minimum update rates needed for the pilot to perceive continuous motion through the instruments.
From stability and control derivatives, constructed software to calculate forces and movements generated by any possible flight control configuration. From an Air Force technical paper, adopted quaternion equations of motion to calculate the dynamic reactions of the simulated aircraft or each simulator iteration cycle. The quaternion model had the advantage of only requiring efficient calculations in addition and subtraction, avoiding costly trigonometric calculations. Obtained from NASA subroutines for simulating any Laplace transformation using difference equations in a digital computer, using basic computer operations to simulate complex mathematical behavior. Using this software combined with table look-ups to simulate nonlinear components, simulated B-1B aircraft dynamic control system responses as well as engine power dynamics. Integrated with hardware and began acceptance testing at the first B-1B base in Dallas.
USAF accepted the flight station design, and awarded several follow-on contracts to support the three interim trainers. SAIC supports those simulators and follow-on simulator designs at Wright-Patterson AFB, that still use much of this software today ported to newer computers and languages. Many experienced B-1B pilots commended how accurate the feel of the simulator was, especially for a procedures trainer.
Rigorous Risk Management Advances Stringent Contract Bid
Charged with risk management as Chief Systems Engineer for a team to bid Distributed Mission Training contract. Client organization in the Air Force responsible for awarding this contract felt that risk management was key to the success of their program. Needed to show an exemplary risk management plan to secure an award. Team (consisting of members of two companies) had no risk management infrastructure.
Adopted a quantitative risk management approach that established objective criteria for probability of occurrence and impact in three key areas: schedule, cost, and technical performance. Enlisted free DoD sponsored tool called “Risk Radar” (MS Access-based). Assembled risk managment tutorial briefing for the team, who brainstormed risk areas and applied the criteria to rank risks for probability and impact of occurrence.
Applied a combined metric of probability and impact to rank risk overall, and divide them into red/yellow/green categories. For red and yellow risks, developed mitigation and contingency plans. For green risks, assigned owners to monitor throughout the program. Built mitigation action plans and specific program processes to mitigate risks. In some cases, added activities such as prototype and test of critical configurations to the work breakdown structure. For red risks, added contingency costs to the program where possible. Reassessed all risks once mitigation and contingency were added, and continued developing plans until red impacts were eliminated.
Government evaluation team approved the plans, and in review awarded high marks for risk management. Risk Radar worked impressively. Objective criteria leading to quantifiable risk assessment enabled aligning a large team of technical and management staff of diverse opinions and perspectives.
Knowledge Transfer Reaps New Business & Expertise
Company division held a contract to provide support to McClellan AFB in Sacramento, which was in the third year of a five-year process to close and transfer the work to other bases. Company provided logistical support for space-based communications while government technical staff provided depot-level maintenance on electronic boards for critical systems. One system required a unique test set developed internally at McClellan to diagnose and repair boards. Because of the closure, and the local experts not wanting to leave CA, the government was about to lose its corporate knowledge to operate the test system and repair the boards. They issued the company an RFP to preserve the capability and keep the systems operating.
Internal company team possessed no knowledge of the test system or organic maintenance procedures. Government employees were looking for jobs and receiving job offers from other organizations. Government organization performing the work was rated as Software Engineering Institute (SEI) Level 3 in software organizational capability maturity model, while SAIC had just evaluated as SEI Level 2, yet needed to show low risk for the project. Additional improvements to the system were under way, and required expertise from the government team.
Offered the government employees work at the existing company division location. One key engineer accepted. Lead engineer on the government side did not, but was sufficiently impressed with team leadership that he assisted in understanding the work and developing an approach. Assembled other staff that could operate the test system and perform repairs. Built a detailed project plan that incorporated all key work, along with key processes to support system maintenance. Sized the software portions of the project according to practices designed to achieve SEI Level 3 rating. Obtained historical basis of estimates from government staff before they left their jobs. Planned the work and applied earned-value tracking to the project. Briefed senior Air Force staff on solidified takeover plans.
Senior staff accountable for continued operation of the satellite communications system was relieved to have a solution that they believed was low risk. Team was awarded $800K contract for the first year of work, and the system was transferred to the company facility in Sacramento. New staff was trained as part of the new program. Supported by the experienced engineer hired from the government, continued depot-level maintenance and performed modifications to support an upgrade, as well. Received follow-on business for several years at an approximate value of $3.8M over five years. Used the project as evidence to achieve SEI Level 3 assessment, which won future contracts.
Adept Forecasting Turns Around Program Predictability
Took over a large program for a major database release when previous program manager exited. Within two months, got program back on track and into final stages of defect removal and release. Product release was tied to another product being modified and released under a different program, which was slipping major milestones weekly. Needed to get predictable milestone dates around the other program to be successful, and was scheduled to brief senior-level VP management about risk involved with product release milestone dates before the next big marketing event in three months.
Other program’s team could not predict its own results. Development manager was inexperienced, appeared “in over her head,” and was operating at a different location. She worked for two different Directors, each reporting to different VPs, each responsible for different product lines. She was reporting detailed numbers, but not reporting a coherent story, nor revealing any root causes that could be dealt with. Both development and project managers appeared to be constrained from “betraying internal problems” to a sister organization.
Declared a crisis to the two Directors and proposed more Director-level help in forecasting the team’s release. Requested a meeting with the Directors and development team to review defect-removal metrics for the project, including: backlog of defects found, defect arrival rate, fix rate, trends and history over time of the project. Director of the development team initiated a three-day focused effort by project and development managers to collect data and plot time histories. Upon review, discovered that trends did not predict a best- and worst-case completion date, and showed similar patterns to prior releases, which predicted a worst-case scenario.
Identified root causes of recent problems. The one key developer was on extended vacation, and three remaining training developers did not have the skills to perform high levels of defect repair on the product. New target date was the best-case scenario, but all the dependent plans were built around the worst-case scenario. Uncoupled the releases to make first product release milestone, and confidently conveyed new product milestone to sales teams (about six weeks later), which was acceptable to all.
First product milestone was achieved on time for marketing event. The other program’s product milestone was reached slightly ahead of advertised date. Neither of these products had released on a predictable schedule before, which stunned the VPs. All involved came away with a better working relationship.
Estimating & Earned Value Expertise Dominate FFP Competition
USAF System Engineering and Technical Analysis (SETA) Program in Germany was a Firm Fixed Price (FFP) Task Order Contract to the US government, and obtained work by bidding technical support tasks at a fixed price that was reasonable but accounted for any risk. Program was paid upon delivery of all work bid.
The technical work was diverse, and understood mainly by a subject matter expert (Principle Investigator—PI): US citizens able to obtain clearance, available in Germany, and usually not skilled at estimating, planning, or cost/schedule control. Because of the contract structure, it took only one bad project to threaten profitability for the year.
Focused stringently on estimates for bids, as well as terms and conditions for task orders, emphasizing historical basis of estimates, whenever possible, and deliverables. Imposed earned value tracking and analysis on all task orders. Recruited a cost/schedule status expert to administer program tracking stateside, and together worked with individual PIs to capture bids in work packages organized around deliverables and objective evidence of progress before submitting proposals to the government. Updated status and tracking and received reports from PIs monthly. Utilized standard one-page Cost/Schedule status sheet reporting earned value analysis (EVA) status, and using the EVA to forecast cost/schedule at contract end, track issues, technical progress, and actions to correct negative variances.
During 3.5 years as Program Manager, the team never overran a single task. Overall profitability doubled, and the team won re-compete easily because of high customer satisfaction and because no other teams could discern how to beat the bids under FFP rules. As business grew, implemented a PI training program and a CSSR to supplement managers with critical attributes.
|