IC design productivity is often thought of as an elusive term. From a design team's perspective, it is a difficult concept to assess due to the fact that IC design projects are diverse in their input, application areas, and design approaches. Due to these project differences, simple measures like number of transistors or die size are unable to accurately depict the true nature of a chip's complexity and consequently do not lend themselves to assessing trends in IC design productivity. In addition, the factors that impact IC design productivity are themselves diverse and linked in a complex web of interdependencies. Key categories of factors impacting IC design productivity include IP reuse, task automation, modeling, methodologies, project management disciplines, designer expertise, and tools and flows. Isolating and measuring these factors is difficult, yet such measurements are the key to any program for improving productivity.
As the old axiom states, "you cannot improve what you cannot measure." A practical methodology for improving productivity requires first identifying actionable and relevant metrics to measure that do not burden the design process or the design team with excessive overhead. Furthermore, comparing multiple projects requires some type of normalization methodology to account for differences in complexity and scope. Factors to include when normalizing project analysis are design size, application area, process technology, frequency, and IP usage.
Possible measurements
Productivity metrics can be divided into two basic categories: design characteristics and resource utilization. Design characteristics provide insight into both the status of the IC design throughout the design process and the overall complexity of the chip. Resource utilization refers to computer and EDA tool resources as well as information about the personnel effort used during the project. To the extent possible, both basic categories of metrics should be captured in an automated fashion throughout the development cycle to minimize the impact on the design team. However, the recording of personnel effort typically poses the biggest challenge for most companies (the criticality of this information to meaningful productivity analysis will be discussed later in this article).
1. Data from the implementation metrics flows from applications to a central data base.
Automatic capture of many metrics is best done within the design environment. Figure 1 models how the data flow for metrics related to the implementation process could be conceptualized within a design environment. At each major step of the flow, the data for the metrics are extracted. Each time a task is invoked, metrics about task execution and the state of the design are put into a database. Each project has an isolated database, and the environment includes a mechanism for extracting information from each project database and storing a summary in a global database. Similar instrumentation of the flow can be established for verification metrics. Tables 1 and 2 respectively list some examples of metrics to be captured for both the implementation and the verification process.
| Task Summary |
Characteristics |
Analysis |
| Project Name |
Number of Instances |
Design TNS |
| Block Name |
Number of Nets |
Group TNS |
| User Name |
Number of Ports |
Design WNS |
| Job Start Time |
Area |
Group WNS |
| Job End Time |
Utilization |
Dynamic Power |
| Job ID |
Height |
Leakage Power |
| Tool Name |
Width |
Total Power |
| Tool Version |
Number of Clocks |
LVS Errors |
| Host Name |
|
DRC Errors |
1. Sample Implementation metrics that can be collected
| Task Summary |
Characteristics |
Analysis |
| Project Name |
Number of Coverage Points |
Number of Bugs Identified |
| Block Name |
Code Line Coverage |
Number od Bugs Fixed |
| User Name |
Code Conditional Coverage |
Number of Passing Tests |
| Job Start Time |
Code Path Coverage |
Number of Failing Nets |
| Job End Time |
Code State Coverage |
Random Seed Value |
| Job ID |
Number of Assertions |
Assertion hit percentage |
| Tool Name |
Lines of Testbench Code |
Testbench Configuration |
| Tool Version |
Lines of RTL Code |
|
2. Sample verification metrics that can be collected
In addition to compute and EDA tool resource utilization, the other major aspect of resource utilization that must be measured is the effort of the people working on the project. This information can provide some of the most interesting insights, yet is often overlooked. Few companies keep detailed records of the amount of time spent by each member of their development teams, aside from overall headcount records. (General records of project headcount typically provide imprecise measurements at best due to variances in work schedules and the fact that people usually enter and leave projects based on fluctuating, undocumented requirements.) However, with only a small amount of overhead, it is possible to capture detailed information about how design teams spend their time
To be most useful, staffing metrics must record the amount of time spent on the design project as well as the type of work completed by each person. This data can be used to understand issues on the current project and for planning future projects. To support use of the data across multiple projects, it is important to use a consistent time-capture process and template.
It is also important to capture the proper context for analyzing staffing data. Specifically, a set of standard project milestones (for example, the completion of the final netlist) should be used. Figure 2 shows an example of a post-project assessment with categorized effort data and a standard set of milestones. The height of each bar represents the amount of effort expended during a particular time period (in this case, each bar represents one week.) Each differently shaded segment of the vertical bars represents a task completed during the week. Project milestones are identified under the time axis to provide the proper context for analysis.
2. The analysis of effort categorization and milestone achievement can be tabulated graphically (source Synopsys Professional Services physical design project data).
|
