The Basics

What is a distributed system? It’s one of those things that’s hard to define without first defining many other things. Here is a “cascading” definition of a distributed system:

A program
is the code you write.
A process
is what you get when you run it.
A message
is used to communicate between processes.
A packet
is a fragment of a message that might travel on a wire.
A protocol
is a formal description of message formats and the rules that two processes must follow in order to exchange those messages.
A network
is the infrastructure that links computers, workstations, terminals, servers, etc. It consists of routers which are connected by communication links.
A component
can be a process or any piece of hardware required to run a process, support communications between processes, store data, etc.
A distributed system
is an application that executes a collection of protocols to coordinate the actions of multiple processes on a network, such that all components cooperate together to perform a single or small set of related tasks.

Why build a distributed system? There are lots of advantages including the ability to connect remote users with remote resources in an open and scalable way. When we say open, we mean each component is continually open to interaction with other components. When we say scalable, we mean the system can easily be altered to accommodate changes in the number of users, resources and computing entities.

Thus, a distributed system can be much larger and more powerful given the combined capabilities of the distributed components, than combinations of stand-alone systems. But it’s not easy – for a distributed system to be useful, it must be reliable. This is a difficult goal to achieve because of the complexity of the interactions between simultaneously running components.

To be truly reliable, a distributed system must have the following characteristics:

* Fault-Tolerant: It can recover from component failures without performing incorrect actions.
* Highly Available: It can restore operations, permitting it to resume providing services even when some components have failed.
* Recoverable: Failed components can restart themselves and rejoin the system, after the cause of failure has been repaired.
* Consistent: The system can coordinate actions by multiple components often in the presence of concurrency and failure. This underlies the ability of a distributed system to act like a non-distributed system.
* Scalable: It can operate correctly even as some aspect of the system is scaled to a larger size. For example, we might increase the size of the network on which the system is running. This increases the frequency of network outages and could degrade a “non-scalable” system. Similarly, we might increase the number of users or servers, or overall load on the system. In a scalable system, this should not have a significant effect.
* Predictable Performance: The ability to provide desired responsiveness in a timely manner.
* Secure: The system authenticates access to data and services [1]

These are high standards, which are challenging to achieve. Probably the most difficult challenge is a distributed system must be able to continue operating correctly even when components fail. This issue is discussed in the following excerpt of an interview with Ken Arnold. Ken is a research scientist at Sun and is one of the original architects of Jini, and was a member of the architectural team that designed CORBA.

Failure is the defining difference between distributed and local programming, so you have to design distributed systems with the expectation of failure. Imagine asking people, “If the probability of something happening is one in 1013, how often would it happen?” Common sense would be to answer, “Never.” That is an infinitely large number in human terms. But if you ask a physicist, she would say, “All the time. In a cubic foot of air, those things happen all the time.”

When you design distributed systems, you have to say, “Failure happens all the time.” So when you design, you design for failure. It is your number one concern. What does designing for failure mean? One classic problem is partial failure. If I send a message to you and then a network failure occurs, there are two possible outcomes. One is that the message got to you, and then the network broke, and I just didn’t get the response. The other is the message never got to you because the network broke before it arrived.

So if I never receive a response, how do I know which of those two results happened? I cannot determine that without eventually finding you. The network has to be repaired or you have to come up, because maybe what happened was not a network failure but you died. How does this change how I design things? For one thing, it puts a multiplier on the value of simplicity. The more things I can do with you, the more things I have to think about recovering from. [2]

Handling failures is an important theme in distributed systems design. Failures fall into two obvious categories: hardware and software. Hardware failures were a dominant concern until the late 80′s, but since then internal hardware reliability has improved enormously. Decreased heat production and power consumption of smaller circuits, reduction of off-chip connections and wiring, and high-quality manufacturing techniques have all played a positive role in improving hardware reliability. Today, problems are most often associated with connections and mechanical devices, i.e., network failures and drive failures.

Software failures are a significant issue in distributed systems. Even with rigorous testing, software bugs account for a substantial fraction of unplanned downtime (estimated at 25-35%). Residual bugs in mature systems can be classified into two main categories [5].

* Heisenbug: A bug that seems to disappear or alter its characteristics when it is observed or researched. A common example is a bug that occurs in a release-mode compile of a program, but not when researched under debug-mode. The name “heisenbug” is a pun on the “Heisenberg uncertainty principle,” a quantum physics term which is commonly (yet inaccurately) used to refer to the way in which observers affect the measurements of the things that they are observing, by the act of observing alone (this is actually the observer effect, and is commonly confused with the Heisenberg uncertainty principle).

* Bohrbug: A bug (named after the Bohr atom model) that, in contrast to a heisenbug, does not disappear or alter its characteristics when it is researched. A Bohrbug typically manifests itself reliably under a well-defined set of conditions. [6]

Heisenbugs tend to be more prevalent in distributed systems than in local systems. One reason for this is the difficulty programmers have in obtaining a coherent and comprehensive view of the interactions of concurrent processes.

Let’s get a little more specific about the types of failures that can occur in a distributed system:

* Halting failures: A component simply stops. There is no way to detect the failure except by timeout: it either stops sending “I’m alive” (heartbeat) messages or fails to respond to requests. Your computer freezing is a halting failure.
* Fail-stop: A halting failure with some kind of notification to other components. A network file server telling its clients it is about to go down is a fail-stop.
* Omission failures: Failure to send/receive messages primarily due to lack of buffering space, which causes a message to be discarded with no notification to either the sender or receiver. This can happen when routers become overloaded.
* Network failures: A network link breaks.
* Network partition failure: A network fragments into two or more disjoint sub-networks within which messages can be sent, but between which messages are lost. This can occur due to a network failure.
* Timing failures: A temporal property of the system is violated. For example, clocks on different computers which are used to coordinate processes are not synchronized; when a message is delayed longer than a threshold period, etc.
* Byzantine failures: This captures several types of faulty behaviors including data corruption or loss, failures caused by malicious programs, etc. [1]

Our goal is to design a distributed system with the characteristics listed above (fault-tolerant, highly available, recoverable, etc.), which means we must design for failure. To design for failure, we must be careful to not make any assumptions about the reliability of the components of a system.

Everyone, when they first build a distributed system, makes the following eight assumptions. These are so well-known in this field that they are commonly referred to as the “8 Fallacies”.

1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn’t change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous. [3]

Latency: the time between initiating a request for data and the beginning of the actual data transfer.
Bandwidth: A measure of the capacity of a communications channel. The higher a channel’s bandwidth, the more information it can carry.
Topology: The different configurations that can be adopted in building networks, such as a ring, bus, star or meshed.
Homogeneous network: A network running a single network protocol.

So How Is It Done?

Building a reliable system that runs over an unreliable communications network seems like an impossible goal. We are forced to deal with uncertainty. A process knows its own state, and it knows what state other processes were in recently. But the processes have no way of knowing each other’s current state. They lack the equivalent of shared memory. They also lack accurate ways to detect failure, or to distinguish a local software/hardware failure from a communication failure.

Distributed systems design is obviously a challenging endeavor. How do we do it when we are not allowed to assume anything, and there are so many complexities? We start by limiting the scope. We will focus on a particular type of distributed systems design, one that uses a client-server model with mostly standard protocols. It turns out that these standard protocols provide considerable help with the low-level details of reliable network communications, which makes our job easier. Let’s start by reviewing client-server technology and the protocols.
In client-server applications, the server provides some service, such as processing database queries or sending out current stock prices. The client uses the service provided by the server, either displaying database query results to the user or making stock purchase recommendations to an investor. The communication that occurs between the client and the server must be reliable. That is, no data can be dropped and it must arrive on the client side in the same order in which the server sent it.

There are many types of servers we encounter in a distributed system. For example, file servers manage disk storage units on which file systems reside. Database servers house databases and make them available to clients. Network name servers implement a mapping between a symbolic name or a service description and a value such as an IP address and port number for a process that provides the service.

In distributed systems, there can be many servers of a particular type, e.g., multiple file servers or multiple network name servers. The term service is used to denote a set of servers of a particular type. We say that a binding occurs when a process that needs to access a service becomes associated with a particular server which provides the service. There are many binding policies that define how a particular server is chosen. For example, the policy could be based on locality (a Unix NIS client starts by looking first for a server on its own machine); or it could be based on load balance (a CICS client is bound in such a way that uniform responsiveness for all clients is attempted).

A distributed service may employ data replication, where a service maintains multiple copies of data to permit local access at multiple locations, or to increase availability when a server process may have crashed. Caching is a related concept and very common in distributed systems. We say a process has cached data if it maintains a copy of the data locally, for quick access if it is needed again. A cache hit is when a request is satisfied from cached data, rather than from the primary service. For example, browsers use document caching to speed up access to frequently used documents.

Caching is similar to replication, but cached data can become stale. Thus, there may need to be a policy for validating a cached data item before using it. If a cache is actively refreshed by the primary service, caching is identical to replication. [1]

As mentioned earlier, the communication between client and server needs to be reliable. You have probably heard of TCP/IP before. The Internet Protocol (IP) suite is the set of communication protocols that allow for communication on the Internet and most commercial networks. The Transmission Control Protocol (TCP) is one of the core protocols of this suite. Using TCP, clients and servers can create connections to one another, over which they can exchange data in packets. The protocol guarantees reliable and in-order delivery of data from sender to receiver.

The IP suite can be viewed as a set of layers, each layer having the property that it only uses the functions of the layer below, and only exports functionality to the layer above. A system that implements protocol behavior consisting of layers is known as a protocol stack. Protocol stacks can be implemented either in hardware or software, or a mixture of both. Typically, only the lower layers are implemented in hardware, with the higher layers being implemented in software.
Resource : The history of TCP/IP mirrors the evolution of the Internet. Here is a brief overview of this history.

There are four layers in the IP suite:

1. Application Layer : The application layer is used by most programs that require network communication. Data is passed down from the program in an application-specific format to the next layer, then encapsulated into a transport layer protocol. Examples of applications are HTTP, FTP or Telnet.

2. Transport Layer : The transport layer’s responsibilities include end-to-end message transfer independent of the underlying network, along with error control, fragmentation and flow control. End-to-end message transmission at the transport layer can be categorized as either connection-oriented (TCP) or connectionless (UDP). TCP is the more sophisticated of the two protocols, providing reliable delivery. First, TCP ensures that the receiving computer is ready to accept data. It uses a three-packet handshake in which both the sender and receiver agree that they are ready to communicate. Second, TCP makes sure that data gets to its destination. If the receiver doesn’t acknowledge a particular packet, TCP automatically retransmits the packet typically three times. If necessary, TCP can also split large packets into smaller ones so that data can travel reliably between source and destination. TCP drops duplicate packets and rearranges packets that arrive out of sequence.

UDP is similar to TCP in that it is a protocol for sending and receiving packets across a network, but with two major differences. First, it is connectionless. This means that one program can send off a load of packets to another, but that’s the end of their relationship. The second might send some back to the first and the first might send some more, but there’s never a solid connection. UDP is also different from TCP in that it doesn’t provide any sort of guarantee that the receiver will receive the packets that are sent in the right order. All that is guaranteed is the packet’s contents. This means it’s a lot faster, because there’s no extra overhead for error-checking above the packet level. For this reason, games often use this protocol. In a game, if one packet for updating a screen position goes missing, the player will just jerk a little. The other packets will simply update the position, and the missing packet – although making the movement a little rougher – won’t change anything.

Although TCP is more reliable than UDP, the protocol is still at risk of failing in many ways. TCP uses acknowledgements and retransmission to detect and repair loss. But it cannot overcome longer communication outages that disconnect the sender and receiver for long enough to defeat the retransmission strategy. The normal maximum disconnection time is between 30 and 90 seconds. TCP could signal a failure and give up when both end-points are fine. This is just one example of how TCP can fail, even though it does provide some mitigating strategies.

3. Network Layer : As originally defined, the Network layer solves the problem of getting packets across a single network. With the advent of the concept of internetworking, additional functionality was added to this layer, namely getting data from a source network to a destination network. This generally involves routing the packet across a network of networks, e.g. the Internet. IP performs the basic task of getting packets of data from source to destination.
4. Link Layer : The link layer deals with the physical transmission of data, and usually involves placing frame headers and trailers on packets for travelling over the physical network and dealing with physical components along the way.

Resource : For more information on the IP Suite, refer to the Wikipedia article.
Remote Procedure Calls

Many distributed systems were built using TCP/IP as the foundation for the communication between components. Over time, an efficient method for clients to interact with servers evolved called RPC, which means remote procedure call. It is a powerful technique based on extending the notion of local procedure calling, so that the called procedure may not exist in the same address space as the calling procedure. The two processes may be on the same system, or they may be on different systems with a network connecting them.

An RPC is similar to a function call. Like a function call, when an RPC is made, the arguments are passed to the remote procedure and the caller waits for a response to be returned. In the illustration below, the client makes a procedure call that sends a request to the server. The client process waits until either a reply is received, or it times out. When the request arrives at the server, it calls a dispatch routine that performs the requested service, and sends the reply to the client. After the RPC call is completed, the client process continues.

Threads are common in RPC-based distributed systems. Each incoming request to a server typically spawns a new thread. A thread in the client typically issues an RPC and then blocks (waits). When the reply is received, the client thread resumes execution.

A programmer writing RPC-based code does three things:

1. Specifies the protocol for client-server communication
2. Develops the client program
3. Develops the server program

The communication protocol is created by stubs generated by a protocol compiler. A stub is a routine that doesn’t actually do much other than declare itself and the parameters it accepts. The stub contains just enough code to allow it to be compiled and linked.

The client and server programs must communicate via the procedures and data types specified in the protocol. The server side registers the procedures that may be called by the client and receives and returns data required for processing. The client side calls the remote procedure, passes any required data and receives the returned data.

Thus, an RPC application uses classes generated by the stub generator to execute an RPC and wait for it to finish. The programmer needs to supply classes on the server side that provide the logic for handling an RPC request.

RPC introduces a set of error cases that are not present in local procedure programming. For example, a binding error can occur when a server is not running when the client is started. Version mismatches occur if a client was compiled against one version of a server, but the server has now been updated to a newer version. A timeout can result from a server crash, network problem, or a problem on a client computer.

Some RPC applications view these types of errors as unrecoverable. Fault-tolerant systems, however, have alternate sources for critical services and fail-over from a primary server to a backup server.

A challenging error-handling case occurs when a client needs to know the outcome of a request in order to take the next step, after failure of a server. This can sometimes result in incorrect actions and results. For example, suppose a client process requests a ticket-selling server to check for a seat in the orchestra section of Carnegie Hall. If it’s available, the server records the request and the sale. But the request fails by timing out. Was the seat available and the sale recorded? Even if there is a backup server to which the request can be re-issued, there is a risk that the client will be sold two tickets, which is an expensive mistake in Carnegie Hall [1].

Here are some common error conditions that need to be handled:

* Network data loss resulting in retransmit: Often, a system tries to achieve ‘at most once’ transmission tries. In the worst case, if duplicate transmissions occur, we try to minimize any damage done by the data being received multiple time.

* Server process crashes during RPC operation: If a server process crashes before it completes its task, the system usually recovers correctly because the client will initiate a retry request once the server has recovered. If the server crashes completing the task but before the RPC reply is sent, duplicate requests sometimes result due to client retries.

* Client process crashes before receiving response: Client is restarted. Server discards response data.

Some Distributed Design Principles
Given what we have covered so far, we can define some fundamental design principles which every distributed system designer and software engineer should know. Some of these may seem obvious, but it will be helpful as we proceed to have a good starting list.

* As Ken Arnold says: “You have to design distributed systems with the expectation of failure.” Avoid making assumptions that any component in the system is in a particular state. A classic error scenario is for a process to send data to a process running on a second machine. The process on the first machine receives some data back and processes it, and then sends the results back to the second machine assuming it is ready to receive. Any number of things could have failed in the interim and the sending process must anticipate these possible failures.

* Explicitly define failure scenarios and identify how likely each one might occur. Make sure your code is thoroughly covered for the most likely ones.

* Both clients and servers must be able to deal with unresponsive senders/receivers.

* Think carefully about how much data you send over the network. Minimize traffic as much as possible.

* Latency is the time between initiating a request for data and the beginning of the actual data transfer. Minimizing latency sometimes comes down to a question of whether you should make many little calls/data transfers or one big call/data transfer. The way to make this decision is to experiment. Do small tests to identify the best compromise.

* Don’t assume that data sent across a network (or even sent from disk to disk in a rack) is the same data when it arrives. If you must be sure, do checksums or validity checks on data to verify that the data has not changed.

* Caches and replication strategies are methods for dealing with state across components. We try to minimize stateful components in distributed systems, but it’s challenging. State is something held in one place on behalf of a process that is in another place, something that cannot be reconstructed by any other component. If it can be reconstructed it’s a cache. Caches can be helpful in mitigating the risks of maintaining state across components. But cached data can become stale, so there may need to be a policy for validating a cached data item before using it.

If a process stores information that can’t be reconstructed, then problems arise. One possible question is, “Are you now a single point of failure?” I have to talk to you now – I can’t talk to anyone else. So what happens if you go down? To deal with this issue, you could be replicated. Replication strategies are also useful in mitigating the risks of maintaining state. But there are challenges here too: What if I talk to one replicant and modify some data, then I talk to another? Is that modification guaranteed to have already arrived at the other? What happens if the network gets partitioned and the replicants can’t talk to each other? Can anybody proceed?

There are a set of tradeoffs in deciding how and where to maintain state, and when to use caches and replication. It’s more difficult to run small tests in these scenarios because of the overhead in setting up the different mechanisms.

* Be sensitive to speed and performance. Take time to determine which parts of your system can have a significant impact on performance: Where are the bottlenecks and why? Devise small tests you can do to evaluate alternatives. Profile and measure to learn more. Talk to your colleagues about these alternatives and your results, and decide on the best solution.

* Acks are expensive and tend to be avoided in distributed systems wherever possible.

* Retransmission is costly. It’s important to experiment so you can tune the delay that prompts a retransmission to be optimal.

Exercises

1. Have you ever encountered a Heisenbug? How did you isolate and fix it?

2. For the different failure types listed above, consider what makes each one difficult for a programmer trying to guard against it. What kinds of processing can be added to a program to deal with these failures?

3. Explain why each of the 8 fallacies is actually a fallacy.

4. Contrast TCP and UDP. Under what circumstances would you choose one over the other?

5. What’s the difference between caching and data replication?

6. What are stubs in an RPC implementation?

7. What are some of the error conditions we need to guard against in a distributed environment that we do not need to worry about in a local programming environment?

8. Why are pointers (references) not usually passed as parameters to a Remote Procedure Call?

9. Here is an interesting problem called partial connectivity that can occur in a distributed environment. Let’s say A and B are systems that need to talk to each other. C is a master that also talks to A and B individually. The communications between A and B fail. C can tell that A and B are both healthy. C tells A to send something to B and waits for this to occur. C has no way of knowing that A cannot talk to B, and thus waits and waits and waits. What diagnostics can you add in your code to deal with this situation?

10. What is the leader-election algorithm? How can it be used in a distributed system?

11. This is the Byzantine Generals problem: Two generals are on hills either side of a valley. They each have an army of 1000 soldiers. In the woods in the valley is an enemy army of 1500 men. If each general attacks alone, his army will lose. If they attack together, they will win. They wish to send messengers through the valley to coordinate when to attack. However, the messengers may get lost or caught in the woods (or brainwashed into delivering different messages). How can they devise a scheme by which they either attack with high probability, or not at all?

References

[1] Birman, Kenneth. Reliable Distributed Systems: Technologies, Web Services and Applications. New York: Springer-Verlag, 2005.

[2] Interview with Ken Arnold

[3] The Eight Fallacies

[4] Wikipedia article on IP Suite

[5] Gray, J. and Reuter, A. Transaction Processing: Concepts and Techniques. San Mateo, CA: Morgan Kaufmann, 1993.

[6] Bohrbugs and Heisenbugs

Use Case Diagrams describe the relationships and dependencies between a group of Use Cases and the Actors participating in the process.

It is important to notice that Use Case Diagrams are not suited to represent the design, and cannot describe the internals of a system. Use Case Diagrams are meant to facilitate the communication with the future users of the system, and with the customer, and are specially helpful to determine the required features the system is to have. Use Case Diagrams tell, what the system should do but do not — and cannot — specify how this is to be achieved.

---

Umbrello UML Modeller showing a Use Case Diagram

---

Class Diagram

Class Diagrams show the different classes that make up a system and how they relate to each other. Class Diagrams are said to be “static” diagrams because they show the classes, along with their methods and attributes as well as the static relationships between them: which classes “know” about which classes or which classes “are part” of another class, but do not show the method calls between them.

---

Umbrello UML Modeller showing a Class Diagram

---

Class

A Class defines the attributes and the methods of a set of objects. All objects of this class (instances of this class) share the same behavior, and have the same set of attributes (each object has its own set). The term “Type” is sometimes used instead of Class, but it is important to mention that these two are not the same, and Type is a more general term.

In UML, Classes are represented by rectangles, with the name of the class, and can also show the attributes and operations of the class in two other “compartments” inside the rectangle.

---

Visual representation of a Class in UML

---

Attributes

In UML, Attributes are shown with at least their name, and can also show their type, initial value and other properties. Attributes can also be displayed with their visibility:

• + Stands for public attributes
• # Stands for protected attributes
• - Stands for private attributes

Operations

Operations (methods) are also displayed with at least their name, and can also show their parameters and return types. Operations can, just as Attributes, display their visibility:

• + Stands for public operations
• # Stands for protected operations
• - Stands for private operations

Templates

Classes can have templates, a value which is used for an unspecified class or type. The template type is specified when a class is initiated (i.e. an object is created). Templates exist in modern C++ and will be introduced in Java 1.5 where they will be called Generics.

Class Associations

Classes can relate (be associated with) to each other in different ways:

Generalization

Inheritance is one of the fundamental concepts of Object Oriented programming, in which a class “gains” all of the attributes and operations of the class it inherits from, and can override/modify some of them, as well as add more attributes and operations of its own.

In UML, a Generalization association between two classes puts them in a hierarchy representing the concept of inheritance of a derived class from a base class. In UML, Generalizations are represented by a line connecting the two classes, with an arrow on the side of the base class.

---

Visual representation of a generalization in UML

---

Associations

An association represents a relationship between classes, and gives the common semantics and structure for many types of “connections” between objects.

Associations are the mechanism that allows objects to communicate to each other. It describes the connection between different classes (the connection between the actual objects is called object connection, or link.

Associations can have a role that specifies the purpose of the association and can be uni- or bidirectional (indicates if the two objects participating in the relationship can send messages to the other, of if only one of them knows about the other). Each end of the association also has a multiplicity value, which dictates how many objects on this side of the association can relate to one object on the other side.

In UML, associations are represented as lines connecting the classes participating in the relationship, and can also show the role and the multiplicity of each of the participants. Multiplicity is displayed as a range [min..max] of non-negative values, with a star (*) on the maximum side representing infinite.

---

Visual representation of an Association in UML

---

Aggregation

Aggregations are a special type of associations in which the two participating classes don’t have an equal status, but make a “whole-part” relationship. An Aggregation describes how the class that takes the role of the whole, is composed (has) of other classes, which take the role of the parts. For Aggregations, the class acting as the whole always has a multiplicity of one.

In UML, Aggregations are represented by an association that shows a rhomb on the side of the whole.

---

Visual representation of an Aggregation relationship in UML

---

Composition

Compositions are associations that represent very strong aggregations. This means, Compositions form whole-part relationships as well, but the relationship is so strong that the parts cannot exist on its own. They exist only inside the whole, and if the whole is destroyed the parts die too.

In UML, Compositions are represented by a solid rhomb on the side of the whole.

---

---

Other Class Diagram Items

Class diagrams can contain several other items besides classes.

Interfaces

Interfaces are abstract classes which means instances can not be directly created of them. They can contain operations but no attributes. Classes can inherit from interfaces (through a realisation association) and instances can then be made of these diagrams.

Datatypes

Datatypes are primitives which are typically built into a programming language. Common examples include integers and booleans. They can not have relationships to classes but classes can have relationships to them.

Enums

Enums are a simple list of values. A typical example is an enum for days of the week. The options of an enum are called Enum Literals. Like datatypes they can not have relationships to classes but classes can have relationships to them.

Packages

Packages represent a namespace in a programming language. In a diagram they are used to represent parts of a system which contain more than one class, maybe hundereds of classes.

Sequence Diagrams

Sequence Diagrams show the message exchange (i.e. method call) between several Objects in a specific time-delimited situation. Objects are instances of classes. Sequence Diagrams put special emphasis in the order and the times in which the messages to the objects are sent.

In Sequence Diagrams objects are represented through vertical dashed lines, with the name of the Object on the top. The time axis is also vertical, increasing downwards, so that messages are sent from one Object to another in the form of arrows with the operation and parameters name.

---

Umbrello UML Modeller showing a Sequence Diagram

---

Messages can be either synchronous, the normal type of message call where control is passed to the called object until that method has finished running, or asynchronous where control is passed back directly to the calling object. Synchronous messages have a vertical box on the side of the called object to show the flow of program control.

Collaboration Diagrams

Collaboration Diagrams show the interactions occurring between the objects participating in a specific situation. This is more or less the same information shown by Sequence Diagrams but there the emphasis is put on how the interactions occur in time while the Collaboration Diagrams put the relationships between the objects and their topology in the foreground.

In Collaboration Diagrams messages sent from one object to another are represented by arrows, showing the message name, parameters, and the sequence of the message. Collaboration Diagrams are specially well suited to showing a specific program flow or situation and are one of the best diagram types to quickly demonstrate or explain one process in the program logic.

---

Umbrello UML Modeller showing a Collaboration Diagram

---

State Diagram

State Diagrams show the different states of an Object during its life and the stimuli that cause the Object to change its state.

State Diagrams view Objects as state machines or finite automates that can be in one of a set of finite states and that can change its state via one of a finite set of stimuli. For example an Object of type NetServer can be in one of following states during its life:

• Ready
• Listening
• Working
• Stopped

and the events that can cause the Object to change states are

• Object is created
• Object receives message listen
• A Client requests a connection over the network
• A Client terminates a request
• The request is executed and terminated
• Object receives message stop
• etc

---

Umbrello UML Modeller showing a State Diagram

---

State

States are the building block of State Diagrams. A State belongs to exactly one class and represents a summary of the values the attributes of a class can take. A UML State describes the internal state of an object of one particular class

Note that not every change in one of the attributes of an object should be represented by a State but only those changes that can significantly affect the workings of the object

There are two special types of States: Start and End. They are special in that there is no event that can cause an Object to return to its Start state, in the same way as there is no event that can possible take an Object out of its End state once it has reached it.

Activity Diagram

Activity Diagrams describe the sequence of activities in a system with the help of Activities. Activity Diagrams are a special form of State Diagrams, that only (or mostly) contains Activities.

---

Umbrello UML Modeller showing an Activity Diagram

---

Activity Diagrams are similar to procedural Flux Diagrams, with the difference that all Activities are clearly attached to Objects.

Activity Diagrams are always associated to a Class, an Operation or a Use Case.

Activity Diagrams support sequential as well as parallel Activities. Parallel execution is represented via Fork/Wait icons, and for the Activities running in parallel, it is not important the order in which they are carried out (they can be executed at the same time or one after the other)

Activity

An Activity is a single step in a process. One Activity is one state in the system with internal activity and, at least, one outgoing transition. Activities can also have more than one outgoing transition if they have different conditions.

Activities can form hierarchies, this means that an Activity can be composed of several “detail” Activities, in which case the incoming and outgoing transitions should match the incoming and outgoing transitions of the detail diagram.

Entity Relationship Diagrams

Entity Relationship Diagrams (ER Diagrams) show the conceptual design of database applications. They depict the various entities (concepts) in the information system and the existing relationships and constraints between them. An extension of Entity Relationship Diagrams named ‘Extended Entity Relationship Diagrams’ or ‘Enhanced Entity Relationship Diagrams’ (EER), are used to incorporate Object Oriented design techniques in ER Diagrams.

---

Umbrello showing an Entity Relationship Diagram

---

Entity

An Entity is any concept in the real world with an independent existence. It may be an object with a physical existence ( example, Computer, Robot) or it may be an object with a conceptual existence ( eq: University Course). Each entity has a set of attributes which describe the properties of the Entity.

Note: No standard notations exist for depicting ER Diagrams. Different texts on this subject use different notations. The concepts and notations for EER diagrams used in Umbrello are from the following book : Elmasri R. and Navathe S. (2004). Fundamentals of Database Systems 4th edn. Addison Wesley

In an ER Diagram, Entities are represented by rectangles, with the name of the entity at the top, and can also show the attributes of the entity in another “compartment” inside the rectangle.

---

Visual representation of an entity in an ER Diagram

---

Entity Attributes

In ER Diagrams , Entity Attributes are shown with their name in a different compartment of the Entity to which they belong.

Constraints

Constraints in ER Diagrams specify the restrictions on data in the information schema.

There are four types of constraints supported in Umbrello :

• Primary Key: The set of attributes declared as primary key are unique to the entity. There can be only one primary key in an Entity and none of its constituent attributes can be NULL.
• Unique Key: The set of attributes declared as unique are unique to the entity. There can be many unique constraints on an Entity. Its constituent attributes can be NULL. Unique Keys and Primary Keys uniquely identify a row in a table ( entity )
• Foreign Key: A Foreign Key is a referential constraint between two tables. The foreign key identifies a column or a set of columns in one (referencing) table that refers to a column or set of columns in another (referenced) table. The columns in the referenced table must form a primary key or unique key.
• Check Constraint: A check constraint (also known as table check constraint) is a condition that defines valid data when adding or updating an entry in a table of a relational database. A check constraint is applied to each row in the table. The constraint must be a predicate. It can refer to a single or multiple columns of the table.

  Example: price >= 0

Extended Entity Relationship (EER) Diagram Concepts

Specialization

Specialization is a way to form new entities using entities that have already been defined. The new entities, known as derived entities, take over (or inherit) attributes of the pre-existing entities, which are referred to as base entities . It is intended to help reuse existing data with little or no modification.

In Umbrello, one can specify Disjoint and Overlapping Specialization

Disjoint Specialization

Disjoint Specialization specifies that the subclasses of the specialization must be disjoint. This means that an entity can be a member of at most one of the derived entities of the specialization

---

Visual representation of Disjoint Specialization in EER Diagram

---

Overlapping Specialization

When the derived entities are not constrained to be disjoint, their set of entities are said to be in overlapping specialization. This means that the same real world entity may be a member of more than one derived entity of the specialization

---

Visual representation of Overlapping Specialization in EER Diagram

---

Category

A derived Entity is said to be a Category when it represents a collection of objects that is a subset of Union of the distinct entity types. A Category is modelled when the need arises for a single superclass/subclass relationship with more than one superclass, where the superclasses represent different entity types. ( Much like multiple inheritance in Object Oriented Programming ).

---

Visual representation of a Category in EER Diagram

In the second half of this chapter, we dipped our toes in the waters of UML— taking a quick look at some of the different types of diagram—but it’ll be time to jump right in during the next chapter.

Exercise Solutions

Solutions to Exercise 101

1. The system creates a kennel assignment, a mapping of a pet to a specific pen.
2. The system closes the kennel assignment.
3. Checking a pet in or checking a pet out.
4. Logging a medical problem.

Solutions to Exercise 102

1. Enter the pet’s name. Find available kennel space. Assign the kennel to the pet.
2. Enter the pet’s personal data. Create a new record for the pet.

Solutions to Exercise 103

1.  ICareGiving.

2.  Reservation Center through the IReservations interface.
Accounting Center through the IAccounting interface.

3.  Intercom, telephone, email, and postal.

Solutions to Exercise 104

1.  Care Giver Center, Vet Record Page, ICareGiving interface,
and ICommunications interface.

2.  Care Giver and Veterinarian.

3. ICareGiving.

4.  Via a telephone contact.

Solutions to Exercise 105

1. Pet Record, Reservation Record, and Kennel Space.
2. Create and Close.
3. Name, Species, Breed, Owner, and Schedule.
4. Kennel Number, Building, Size, and Status.

Solutions to Exercise 106

1. Specifications Completed.
2. Construction Completed (from Defined), Pet Checked Out (from In Use), and Pet Relocated (also from In Use).
3. Deconstructed.

4. Via a Pet Checked Out event or a Pet Relocated event,
followed by a Dismantled event. This prevents the system from
having a pet with no defined pen. (Can’t have the dogs running
loose!)

Solutions to Exercise 107

1. Accounting center and reception center.
2. Via the modem.
3. Via the Internet.
4. According to this diagram, the only path possible is through the KMS server. Although the diagram doesn’t explicitly state this, the likely approach is that the care giver station updates information in the central database, and the reception center reads this updated data.

Solutions to Exercise 108

1. Care Giver Classes, Accounting Classes, Reservation Classes, and KMS Central Classes.
2. KMS Interfaces and KMS Database Classes.

2. Steve McConnell, Code Complete: A Practical Handbook of
Software Construction (Microsoft Press, 1993), pp. 81–93, 116–
130, 150. McConnell provides far more information on code
design than I can cover here.

3. See http://www.cuttysark.org.uk/ for pictures and the history
of this vessel.

4. Steve McConnell, Software Project Survival Guide (Microsoft
Press, 1997), p. 29

5. Kent Beck, Extreme Programming Explained: Embrace Change
(Addison Wesley, 1999), pp. 21–25

6. In December 2002, Rational became a division of IBM Corporation.

7. Martin Fowler and Kendall Scott, UML Distilled, Second Edition
(Addison-Wesley, 1999), pp. 13–38

8. Murray R. Cantor, Object-Oriented Project Management with UML
(John Wiley & Sons, 1998), pp. 98–103

9. Putnam P. Texel and Charles B. Williams, Use Cases Combined
with Booch/OMT/UML: Process and Products (Prentice Hall,
1997), pp. 3–9ff.

10. Ivar Jacoboson, Grady Booch, and James Rumbaugh, The Unified
Software Development Process (Addison-Wesley, 1999)

11. Scott W. Ambler, Agile Modeling: Effective Practices for eXtreme
Programming and the Unified Process (John Wiley & Sons, 2002)

12. Klutz Press Editors, Draw the Marvel Comics Super Heroes (Klutz
Press, 1995), pp. 20–32

13. Scott Adams, The Dilbert Principle (HarperBusiness, 1997), p. 324

Don’t worry about being perfect all at once (or ever!). If you’re communicating, you’re using UML effectively. There’s always room for improvement; but don’t let imperfection stop you from progress. That’s a key point in learning and applying UML and in applying UML as part of a process. Your designs won’t be perfect; but as your team reviews them and you refine them, they will become good enough: good enough to build the system, good enough to guide the testing and the documentation, good enough to drive the management of the process. A team that waits for perfection is just as bad as a team that is wedded to code-and-fix: neither team produces an effective design that leads to a successful system. Remember: code-and-fix bad; design-and-fix good.

Letting Go of Perfection: A Lesson from the Graphic Arts

If I haven’t persuaded you yet to let go of perfection, it won’t surprise me. It’s easy for me to say that you can make imperfect diagrams and improve them later; but how do you do it? How do you just sit down and start drawing diagrams when there’s nothing you know? I hear this most often from students who are drawing Sequence or Activity Diagrams: “How can I draw this diagram until I know which objects are in the system?” And they have it exactly backwards: they’ll use the diagrams to discover objects that solve the problem. But they want to draw the right picture. Maybe they’re afraid to look foolish in reviews. (Reviews can be intimidating even to a strong ego like mine!) Maybe they’re just hypersensitive to the risks in the wrong design. Maybe they have embraced the idea that the design is supposed to help them get things right, and therefore are reluctant to risk getting things wrong. But whatever the reason, they just can’t seem to let go.

So I try a different approach. My students who have seen me sketch diagrams at the flip chart are very aware that one thing I’m not is an artist. But borrowing a technique from the graphic arts,¹² I draw the images shown in Figure 1-10, one on top of another.

Figure 1-10. Refining from imperfect to communication

The technique is simple: put down some detail—even if it’s wrong, such as the sizing circles in the first image —to serve as a basis for further development; then add and erase detail as needed to refine the picture. I’m still not an artist (I draw fencers because the hard parts are hidden behind a mask and a weapon); but by applying some simple techniques and refining, I end up with a much more recognizable picture than I would have if I sat down and tried to draw a perfect fencer from scratch. Imperfection and refinement produces better (and faster!) results than does a foolish insistence on perfection.

Scott Adams tells us, “Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.”¹³ Take this attitude to heart in your analysis and design process.

Figure 1-9. Package Diagram of the Kennel Management System

Exercise 108:  Reading a Package Diagram

Answer the following questions about the diagram:

1. Which packages make use of information from the KMS Interfaces package?
2. Which packages does the KMS Central Classes package make use of?

An Apology to Database Modelers and Data Flow Diagrammers

If you come from the database-modeling world and are used to entity relationship diagrams (ERDs), you probably got Exercise 108 exactly wrong. The same is probably true if you come from the world of data flow diagrams (DFDs). In both of those diagramming techniques, the arrows on a diagram indicate the direction of data flow. In UML, the dependency arrows (- – - – >) indicate the direction of knowledge and control. To those people familiar with ERDs and DFDs, this seems exactly backwards. Their instinct is to interpret Figure 1-9 as meaning that KMS Interfaces make use of Care Giver Classes, Accounting Classes, Reservation Classes, and KMS Central Classes.

The UML convention isn’t quite as backwards as it seems. How would a class in Care Giver Classes make use of an interface in KMS Interfaces? By calling its methods, of course. And very likely, the class would pass data to the interface as well. So control and even some data do indeed flow in the direction of the dependency arrow.

And if that doesn’t help you to keep the meaning of the arrows straight, I can only apologize, and claim innocence: I didn’t write the UML (though its convention seems “correct” in my biased perspective). I do know that this is a source of confusion for ERD and DFD practitioners; and I hope that this warning will help to minimize your confusion.

Figure 1-4. Component Diagram of the Kennel Management System

Exercise 103:  Reading a Component Diagram

Answer the following questions about the diagram:

1. The Care Giver Center is the Web page that the care giver uses to enter information about a pet. What interface does it use to provide data to the KMS?
2. What other components provide data to the KMS, and through what interfaces?
3. What types of contacts can be made through the Comm Center component?

A Sequence Diagram depicts the detailed behavior over time within one path or scenario of a single functional requirement. It’s useful for understanding the flow of messages between elements of your system. Figure 1-5 presents an example of a Sequence Diagram for the Kennel Management System.

Figure 1-5. Sequence Diagram for creating a vet record

Exercise 104:  Reading a Sequence Diagram

Answer the following questions about the diagram:

1. Which objects in the system are involved in creating a vet record?
2. Which actors outside the system are involved in creating a vet record?
3. What interface does the Vet Record Page use for creating a vet record?
4. How does the system contact the veterinarian?

A Class Diagram depicts the classes and interfaces within the design of your system, as well as the relations between them. It’s useful for defining the internal, Object-Oriented structure of your code. Figure 1-6 presents an example Class Diagram from the Kennel Management System.

Figure 1-6. Class Diagram for Kennel assignment

Answer the following questions about the diagram:

1. What three classes are associated with the Kennel Assignment class?
2. What operations can objects of the Kennel Assignment class perform?
3. What attributes describe an object of the Pet Record class?
4. What attributes describe an object of the Kennel Space class?

A Statechart Diagram depicts how the state of your system changes in response to internal and external events. It’s useful for ensuring that each event is handled properly no matter what state your system may be in. Figure 1-7 presents an example of a Statechart Diagram from the Kennel Management System. This diagram illustrates events involving the availability of kennel spaces.

Figure 1-7. Statechart Diagram for Kennel Spaces

Exercise 106:  Reading a Statechart Diagram

Answer the following questions about the diagram:

1. What event causes a kennel space, or pen, to enter the Defined state?
2. What events (from which states) cause a pen to enter the Available state?
3. What state does a pen enter when it’s currently in the Available state, and a Dismantled event occurs?
4. How can a pen go from the In Use state to the Deconstructed state?

A Deployment Diagram depicts how the deployable units of your system—applications, components, data stores, etc.—are assigned to various nodes, as well as how the nodes communicate with each other and with devices. It’s useful both as a map of your system and as a means for studying the load across your system. Figure 1-8 presents a simple example of a Deployment Diagram for the Kennel Management System.

Figure 1-8. Deployment Diagram for the Kennel Management System

Exercise 107:  Reading a Deployment Diagram

Answer the following questions about the diagram:

1. Which processes are running on the reception station?
2. How is the KMS server connected to the telephone?
3. How does the owner’s PC access pet information on the KMS server?
4. How does information go from the care giver station to the reception station?

• Use Case Diagrams show the externally visible behavior of the system. You’ll see these in the next chapter (Chapter 2) and later on when we look at Step 1 of Five-Step UML in Chapter 6.
• Activity Diagrams show an elaboration of the behavior of the system. You’ll see these in the next chapter, and use them during Step 2 of Five- Step UML in Chapter 7. A recent addition to UML is the division of Activity Diagrams into swimlanes, which you’ll see in the next chapter, and we’ll use during Step 3 of Five-Step UML in Chapter 8.
• Component Diagrams show architecture of the system. You’ll see these in the next chapter, and we’ll use them during Step 4 of Five-Step UML in Chapter 9.
• Sequence Diagrams show object interactions over time. We don’t use these diagrams as part of Five-Step UML, but we’ll look at them in Chapter 13.
• Collaboration Diagrams show object interactions with emphasis on relations between objects. We don’t use this type of diagram as part of Five-Step UML, but we’ll look at them in Chapter 13.
• Class Diagrams show class definition and relations. You’ll see these in the next chapter, and we’ll use them during Step 5 of Five-Step UML in Chapter 10.
• Statechart Diagrams show state changes in response to events. We don’t use these diagrams as part of Five-Step UML, but we’ll talk about them in Chapter 13.
• Deployment Diagrams show physical architecture of the system. We’ll use these in Chapter 11.
• Package Diagrams show the hierarchical structure of your design. These can be useful for organizing many different types of elements and you’ll be seeing this type of diagram often throughout the book.

I have a secret. You know those different UML diagram types? Does it annoy you just a bit, having to learn so many new notations to use UML? Does it make you feel like skipping OOAD and just jumping into code?

Well, it’s true that making good UML diagrams takes some skill and some practice; but reading well-crafted diagrams is a very different matter. Just between you and me, I’ll bet you already know how to read UML diagrams, and you don’t even know it. For the rest of this chapter, I’m going to show you some examples of the different UML diagrams, and ask you a few simple questions about what it is they are saying. Don’t worry if you can’t follow all of the notation at this stage—or indeed if you find the questions too simplistic—the point of the following exercises is simply to show you how straightforward it is to read UML diagrams, and how you’ll be able to answer questions about them right away. You won’t be an expert by any measure—that comes in later chapters—but you’ll be able to understand the information contained in the diagrams.

Because, see, here’s the secret: UML—it’s all about communication . . .

The exercises and examples in this book all derive from the Kennel Management System (KMS) for Sandy’s Pets, a high-tech kennel providing temporary and long-term care for cats, dogs, birds, and exotic pets. The Kennel Management System must provide familiar features tailored to the pet care domain:

• Reservations and occupancy. Like a good hotel, the KMS must allow pet owners to reserve kennel space (i.e., “pens”) and to check pets in and out. Unlike a hotel, a significant fraction of the occupants reside in the kennel long term or even full time.
• Exercise and grooming schedule. Like a good physical therapy clinic, the KMS must support and maintain exercise and grooming schedules for each resident. Unlike a clinic, these schedules are dictated by the pet owner rather than by a physician or therapist.
• Nutrition and dietetics. Like a good health spa, the KMS must support both standard and customized diets for each resident. Unlike a health spa, some of the residents only eat live food.
• Inventory and ordering. Like a good restaurant, the KMS must keep food (and other supplies) on hand to meet the needs of a varied clientele. Unlike most restaurants (and as noted previously), some of the food must be stored live.
• Surveillance and tracking. Like a good day care center, the KMS must ensure that its residents are safe and secure, including allowing the pet owners to view their pets via Web cams. Unlike day care centers, each resident will be equipped with a computerized collar or tag, which will allow sensors in the kennel to locate and check the status of each pet.

• Health care and medication. Like a good health care management system, the KMS must schedule regular and emergency medical visits, maintain a medical history, and manage dispensing of medications. Unlike typical health care systems, the residents come from a wide variety of species and thus need species-specific medications and treatment programs.
• Customer relations and pedigrees. Like a good contact management system, the KMS must track information about residents past, present, and possibly future. Unlike typical contact management systems, the KMS must maintain information about both a pet’s parentage and future breeding plans.

The KMS must also provide basic human resources, accounting, and administration functions. For a more detailed specification of the Kennel Management System, see Appendix A.

A Use Case Diagram depicts actions by people and systems outside your system, along with what your system does in response. It’s useful for depicting the functional requirements of your system. Figure 1-2 shows an example of a simple Use Case Diagram for the Kennel Management System.

Figure 1-2. Use Case Diagram for care giver requirements

Excercise 101:  Reading a Use Case Diagram

Answer the following questions about the diagram:

1. What does the KMS do when the care giver checks a pet in?
2. What does the KMS do when the care giver checks a pet out?
3. What action or actions by the care giver will cause the KMS to update a kennel assignment?
4. What action or actions by the Care Giver will involve the Veterinarian?

An Activity Diagram depicts the detailed behavior inside a single functional requirement, including a primary scenario and a number of alternate scenarios. It’s useful for ensuring that you thoroughly understand a given functionality. Figure 1-3 presents an Activity Diagram for the one functional requirement of the Kennel Management System for Sandy’s Pets.

Figure 1-3. Activity Diagram for assigning a pen to a pet

Exercise 102: Reading an Activity Diagram

Answer the following questions about the diagram:

1. What is the sequence of steps the system performs when assigning a pen to a pet that has visited previously?
2. What additional steps are performed for a pet that is new to the kennel?

What Makes the UML Unified?

To answer that, you need to jump in the WayBack Machine and travel back to the ancient history of OOAD—say, 1995. At that time, Object-Oriented Programming had proven its worth as a way of building applications and systems; and the hot new topics in the OO world were OOA and OOD. Since these topics were larger in scope than mere programming, the practitioners needed a way of displaying large, complex concepts in a simple graphical fashion. A number of competing OO notations emerged, chief among them being the Booch Notation designed by Grady Booch and the Object Modeling Technique (OMT) designed by James Rumbaugh (and others). And then began the religious wars: fanatical adherents of the competing notations fought petty little battles over clouds (Booch’s symbol for a class) versus rectangles (OMT’s symbol for a class).

Booch and Rumbaugh looked on this confusion with dismay: they saw the differences between their notations as minor, unworthy of so much rancor. And worse, they saw the religious wars detracting from what they felt should be the new focus: the OOAD process itself, particularly as a means for capturing requirements and behavior. They were very impressed by some of Ivar Jacobson’s work on Sequence Diagrams and his Objectory methodology; but with all the shouting about notation, no one was talking about process. So Booch and Rumbaugh and Jacobson (the Three Amigos) went on retreat to hammer out differences in their notations in private, and to adopt other useful notations as well (chief among these being David Harel’s State Diagram notation). They emerged with the Unified Modeling Language 0.8, a response to the Object Management Group’s call for a standard object modeling notation; and Booch’s company, Rational⁶ (which had hired Rumbaugh and Jacobson) incorporated UML into Rational ROSE, their OOAD tool. The UML then went through a few cycles of response and revision by the OO community; and UML 1.1 was adopted as a standard by the OMG in 1997. UML has been further refined, and is currently at version 1.4, with version 2.0 on the near horizon.

What UML Isn’t

With so much in UML, it’s worth mentioning what’s not UML. The following sections describe some related concepts that are often confused with UML itself.

A Process

Remember the goal of the Three Amigos: to focus attention on the OOAD process, not on the notation. Their notation isn’t a process in itself; rather, it’s designed to support a wide range of OO processes. There are a number of UML-based OOAD processes, including Fowler’s outline process,⁷ controlled iteration,⁸ Texel and Williams’s complete OO process,⁹ and, of course, the Unified Process (formerly Objectory) from the Three Amigos.¹⁰ A full list would be much longer, but would have a lot in common with these prominent examples. These processes differ somewhat in the degree of formality and the order of operations; but all involve using UML to identify and refine requirements, allocate those requirements to functional modules, and refine those modules. Without a process, new UML students are often adrift in a sea of concepts, with nothing to guide their course.

In this book, you’ll learn UML within the framework of Five-Step UML, which I find to be a “just enough” process: just enough process to help you understand the purpose of UML, but not so much as to bury you in paperwork and obscure the benefits of UML. Five-Step UML isn’t a large, full-featured OOAD process, falling somewhere between an academic exercise and Ambler’s Agile Modeling.¹¹ Still, I find that some particularly process-averse teams that tend to reject a more formal process will accept Five-Step UML.

In the next chapter, I’ll show you how the Five-Step process works, and along the way, you should pick up a broad working knowledge of UML. In Chapter 3, I’ll talk about some pragmatic rules I like to apply to the Five-Step (and indeed any other) modeling process. Throughout the core chapters of this book (Part Two), we’ll work through each of the five steps in more detail, applying it to a real-world case study. Although the focus of this book is on Five-Step UML, we will look in more detail at some other processes and how they work in Chapter 12.

Rational XDE (or Any Other Tool)

Many UML practitioners—and of course, many UML tool vendors—tend to blur the line between UML and a given tool. I myself am prone to equating “what XDE can do” with “what UML can do,” since I do most of my UML work with Rational’s XDE tool. This habit isn’t inherently bad, since it’s usually easier to work with your tools than against them; but it’s important to realize that the features and capabilities of any given tool—even the market-leading UML tool from the people who gave us UML—may differ from the features and capabilities of UML itself. If UML is a language, then every tool speaks with a unique accent.

You can find a list of available UML tools in Appendix B.

A Silver Bullet

Good code will still require blood, sweat, tears, pizza, good management, and lots of thought. UML will help you to organize these factors (except the pizza), but you won’t be able to avoid them.

Object-Oriented Analysis (OOA), then, is the process of defining the problem in terms of objects: real-world objects with which the system must interact, and candidate software objects used to explore various solution alternatives. The natural fit of programming objects to real-world objects has a big impact here: you can define all of your real-world objects in terms of their classes, attributes, and operations.

Design

If analysis means defining the problem, then design is the process of defining the solution. It involves defining the ways in which the system satisfies each of the requirements identified during analysis.

Object-Oriented Design (OOD), then, is the process of defining the components, interfaces, objects, classes, attributes, and operations that will satisfy the requirements. You typically start with the candidate objects defined during analysis, but add much more rigor to their definitions. Then you add or change objects as needed to refine a solution. In large systems, design usually occurs at two scales: architectural design, defining the components from which the system is composed; and component design, defining the classes and interfaces within a component.

Models

Did you ever build a model ship? When I was young, my mom and I built a model of the famous clipper ship Cutty Sark.³ I’ve always been fascinated by the tall ships of old; but I really learned about how they work when we built that model. All of the strange nautical terminology from the old pirate movies—forecastle, capstan, main mast, and especially belaying pins (“You mean they’re not just there so somebody can pull one out and swing it as a club?”)—gained concrete meaning when I assembled them and saw them in the context of the entire system.

Well, that’s a central goal of using UML in OOAD: to let you study and understand a system via a model of that system. Like aerodynamic engineers, construction architects, and others in the physical engineering world, you’ll build models of systems yet to be built, not just models of existing systems. Your models will let you explore design alternatives and test your understanding of the system at a much faster rate and much lower cost than the rate and cost associated with actually building the system.

“But wait a minute!” the skeptic in the crowd shouts. “I can see the Cutty Sark, if I travel to London. And I can see the model Cutty Sark, if I visit your home. I can look at the two, and see how the model relates to the real thing. But I can’t ‘look at’ software, except at its user interface. So your model looks like the UI? Isn’t that just a prototype?” That’s the problem with the usual engineering model analogy as applied to software models: there’s no direct physical correspondence between the model and the final product. A better analogy is to models in less tangible scientific disciplines. Quantum physics is a good example of a field in which models help you to understand things you can’t see: no one can see quarks or leptons or hadrons or any of the many subatomic particles; and attributes like charm and strangeness have only metaphorical meaning. The models of quantum physics aren’t literally true, yet they’re very useful in understanding physical phenomena. Software models are like that: useful metaphors and abstractions to help you think about a problem and a solution, not literal depictions of the code itself.

In the case of OOAD with UML, your models consist primarily of diagrams: static diagrams that depict the structure of the system, and dynamic diagrams that depict the behavior of the system. With the dynamic diagrams, you can trace through the behavior and analyze how various scenarios play out. With the static diagrams, you can ensure that each component or class has access to the interfaces and information that it needs to carry out its responsibilities. And it’s very easy to make changes in these models: adding or moving or deleting a line takes moments; and reviewing the change in a diagram takes minutes. Contrast that with actually building the code: hours to implement the change, hours more to test and review it.

But remember The Model Rule:

To use UML effectively, you should never be simply drawing pretty pictures; you should always be editing an underlying model, using the pretty pictures as your user interface.

Your core artifact of the OOAD process is the model. In fact, you will likely have multiple models:

• Analysis Model. This is a model of the existing system, the end user’s requirements, and a high-level understanding of a possible solution to those requirements.
• Architecture Model. This is an evolving model of the structure of the solution to the requirements defined in the Analysis Model. Its primary focus is on the architecture: the components, interfaces, and structure of the solution; the deployment of that structure across nodes; and the trade-offs and decisions that led up to that structure.
• Component (Design) Models. This is a number of models (roughly, one per component) that depict the internal structure of the pieces of the Architecture Model. Each Component Model focuses on the detailed class structure of its component, and allows the design team to precisely specify the attributes, operations, dependencies, and behaviors of its classes.

Depending on your development process, you may have even more models: a Business Model, a Domain Model, possibly others. The major benefit of models is that you can make model changes far earlier in the development cycle than you can make code changes, and far easier. And because you can make changes early, you can make your mistakes early. And that’s a good thing, because early detection and correction is cheap detection and correction. Modeling will let you catch your most costly bugs early; and early detection and correction can save you a factor of 50 to 200 on the cost and schedule of a bug fix.⁴

Equal Time for an Extreme Perspective

Although software engineering literature is rife with research that demonstrates that the cost to fix a defect rises catastrophically over time, those in the Extreme Programming camp disagree. They argue that all such research is dated, and that modern development tools and techniques allow a maximum limit to the cost of change.⁵ They see not an exponential growth, but an asymptotic approach, as shown in Figure 1-1.

Figure 1-1. Cost to correct defects over time

To which theory I can only reply, “So?”

• First, their theory is a prediction, not a guarantee: if you apply the techniques they advocate and if you do everything correctly and if you are working in the right sort of environment on the right sort of project, then the lessons of 30 years of software engineering will allow you to attain the asymptotic curve. Good for you, if you can do that; but the exponential curve is the default behavior. (Furthermore, I would argue that modeling is a useful tool for attaining that asymptotic curve.)
• Second, look at those curves. Do you want to argue over which curve is “right” at the 8-weeks mark? Or would you rather just fix the problem at the 1-week mark? (Beck argues that there is benefit in waiting for the latest possible time, so that you have the maximum information when you make the fix and also so that you don’t waste time fixing problems that never really arise; but his actual example shows the benefits of making a change as soon as you are aware of it. These concerns aren’t incompatible.)

It took me years to appreciate my friend’s study skills. The process of outlining the text forced him to focus: he had a concrete goal in mind, and wouldn’t allow distractions until he finished it. Furthermore, the outlining involved more of his brain: in order to produce an outline of the textbook, he first had to comprehend the textbook; and then to actually write the outline in his own words, he had to involve yet more areas of his brain, as he had to formulate what he wanted to write. Heck, he even had to involve his tactile and motor areas of his brain in order to write the words. He had a much more fully involved brain, and thus he much more fully understood the material.

This is a benefit you can gain from modeling: by listening and talking and reading and by then translating what you learn into a model, you learn it more thoroughly. This is even more important in system design than in studying from a textbook: in a textbook, you expect that the text will be more or less consistent, and that any seeming inconsistency is most likely a sign of something you misunderstood; but in requirements gathering, contradiction and inconsistency are inevitable parts of the process. By modeling the requirements, you may highlight the inconsistencies, particularly when you gather details from many different parts of a specification and put them together in the same diagram.

And there’s one more modeling benefit, one I believe I may have mentioned earlier: communication. It’s easier to communicate with models than with text, or even with code; and you can communicate with a wider audience with models than with other forms of expression. Once you’ve created a diagram “in your own words,” I can more easily comprehend how you comprehend things by comparing your diagram with what I comprehend.

UML communicates these aspects from the particular perspective of Object Orientation (OO), in which everything is described in terms of objects: the actions that objects take, the relationships between the objects, the deployment of the objects, and the way the states of the objects change in response to external events.

The starting point in this chapter will be an overview of Object-Oriented Analysis and Design (OOAD), focusing in on the three most important concepts it encompasses—objects, analysis, design—because to understand UML, you first must understand these broader concepts. If you’ve programmed with any OO language, then you’re probably already familiar with a lot of these ideas, so I’ll keep this discussion brief. Besides, a full discussion of OOAD is beyond the scope of this book. If you want to explore OOAD further, you should read Booch’s Object-Oriented Analysis and Design with Applications.¹

Next, I’ll discuss the results of the OOAD process, namely, a model. I’ll take a bit of a diversion to discuss the nature of models, how you use them, and why they’re important.

For the rest of the chapter, I’ll be focusing on UML, looking at what it is, and— perhaps more importantly—what it isn’t. But before I get started into the nitty gritty of all the different UML elements and diagrams and what the notation looks like (I’ll save that for the next chapter!), I’ll be showing you some UML diagrams from the case study that I’ll be developing in detail in the next part of the book. Now these diagrams aren’t there to scare you off: quite the contrary. When you start to look at some real-world UML, you’ll see how intuitive it is, and how you can understand much of it without any formal teaching.

Objects

Many modern programming languages depend largely or exclusively on the concept of objects: a close syntactic binding of data to the operations that can be performed upon that data. In these Object-Oriented languages—C++, C#, Java, Eiffel, Smalltalk, Visual Basic .NET, Perl, and many others—programmers create classes, each of which defines the behavior and structure of a number of similar objects. Then they write code that creates and manipulates objects that are instances of those classes.

One reason why objects are a powerful programming technique—the reason most often touted in the early literature on Object-Oriented Programming— is that programmatic objects map naturally to real-world objects. Suppose, for example, that your company has to deal with orders. These orders would probably have an ID number and contain information on products. You could create Order objects, which would map to these real-world objects, and which would have properties such as ID and ProductList. You’d probably want to be able to add a product to the order and to submit the order, so you could write AddProduct and SubmitOrder methods. This mapping between objects in the real world and more abstract code objects encourages programmers to think in the problem domain, rather than in computer science terms. This benefit has perhaps been overstated, however; unless you’re building a simulator of a real-world process, such surrogate “real-world” objects form just the surface of your system. The complexity of your design lies underneath that surface, in code that reflects business rules, resource allocation, algorithms, and other computer science concerns. If you only use objects to reflect the real world, you leave yourself with a lot of work.

A more important benefit of classes and objects is that they form a nice syntactic mechanism for achieving some classic aspects of well-designed code:²

Encapsulation. The goal of encapsulation is to expose only enough of a module or subsystem to allow other modules to make use of it. Object-Oriented Programming allows you to specify the degree of visibility of elements of your code, so that client code is restricted in what it can access. Thus, you can syntactically seal off implementation details, leading to more flexibility and maintainability in your system.

Loose coupling. Coupling refers to the ways in which and degrees to which one part of the system relies on the details of another part. The tighter the coupling, the more changes in one part of the system will ripple throughout the system. With loose coupling, the interfaces between subsystems are well defined and restricted. What lies beyond those interfaces can change without any changes needed in the client sub systems. Object-Oriented Programming supports loose coupling by allowing you to define and publish a class’s methods without publishing how those methods are carried out. This principle goes even further in OO languages that support interfaces (described later in this section).

Strong cohesion. Cohesion refers to the degree in which elements within a subsystem form a single, unified concept, with no excess elements. Where there is high cohesion, there is easier comprehension and thus more reliable code. Object-Oriented Programming supports strong cohesion by allowing you to design classes in which the data and the functions that operate on them are tightly bound together.

Does OO force you to have these quality attributes in your code? I wish! No matter the language, you can write shoddy code with no encapsulation, pathological coupling, and no cohesion. Furthermore, some OO languages are less rigid than others in how much they require you to design around objects. But OO languages certainly support these quality attributes if you take the time to pursue them.

The key concepts in Object-Oriented Programming are these:

Classes. A class is the definition of the behavior and properties of one or more objects within your system. A class binds the data (attributes) of an object to the behavior (operations) that it can perform.

Attributes. An attribute is a data value or state that describes an object and helps you to tell one object from another of the same class. It seems that every new OO language author feels the need to distinguish their language by coming up with new terminology. In some OO languages, these data values are called properties or member variables or member data; but in UML, the proper term is attributes.

Operations. An operation is a behavior or function that an object can perform. Depending on the OO language, these might be called methods or member functions or even messages. The last term, messages, comes from Smalltalk, one of the earliest OO languages, in which all objects communicated by sending messages to each other. You’ll see a similar use of the term message when we study Sequence Diagrams.

Objects. An object is an instance or specific example of a class. If Dog is the class, then Betsy, Ladi, Patches, Jake, Radar, and Frosty are specific instances of the class found in my house. The attributes of the class have specific values within an object of that class; and the operations of a class operate on the attributes of individual objects.

Inheritance. This concept indicates that one class (the superclass) provides some common or general behavior inherited by one or more specific classes (the subclasses). The subclasses then provide more or different behavior beyond that defined in the superclass. For example, besides the Dogs, I have Cat objects and Horse objects that live on my property. Each class has unique behaviors: Dogs must be walked, Cats use the litter box, and Horses drop manure that must be scooped up and thrown in the manure pile. Yet all classes have some common behavior: they must be fed, and they must have vet visits. So I can define a superclass, Pet, and have my subclasses, Dog, Cat, and Horse, derive their shared behavior from the Pet class. In UML, this concept is known under the slightly different term of generalization, in which a superclass provides the generalized behavior of the subclasses. It’s really the same concept, but just looking at it the other way up.

Components. A component is a collection of related classes that together provide a larger set of services. Components in your system might include applications, libraries, ActiveX controls, JavaBeans, daemons, and services. In the .NET environment, most of your projects will require component development.

Interfaces. An interface is a definition of a set of services provided by a component or by a class. This allows further encapsulation: the author of a component can publish just the interfaces to the component, completely hiding any implementation details.

Each of these concepts will be explored in more detail as I discuss the UML diagrams that represent them.