Distributed Agile Development

Team Structures in Onshore-Offshore Distributed Agile Development Teams (OODT)

One of the biggest challenges of offshore development is finding a reliable partner. Unless the US company owns the operations overseas and regardless of the engagement model, special care should be taken to start with a competent and trustworthy partner. If the offshore partner or its employees have prior US work history and some Agile development experience, this will increase the chances of success for the OODT Agile initiative.
Flexibility is another important attribute and despite flashy marketing materials, not all of the offshore companies have developed habits of business savvy and flexibility.
Try to identify an offshore partner willing to work flexible hours to accommodate for the time zone differences. Activities such as testing and deployment can be seen as separate and therefore can be run on a separate schedule, but that does not scale well within the Agile team, especially if it practices the Whole Team. Therefore, effort should be taken to make the whole offshore team available to the onshore team for a portion of the day.

Now, let’s take a look at some possible variations of the offshoring model. There are probably as many models as thinking heads, however it is possible to classify them as partial and complete offshoring. Before looking into the details of each model, let’s take the composition of the typical colocated Agile team as a baseline and compare it to three possible models for offshoring.

Baseline: Colocated Onshore Agile Team

Let’s start with a typical colocated team where all team members are located in a US-based facility. The diagram below is presented as a baseline for comparison with the three possible offshoring models.

Model 1: Partial Offshoring

This is the first model of offshoring where parts of the functional team are shipped overseas. In this model the design team is located onshore and it sends the requirements overseas. The offshore team responds with back fully-tested and working code.The onshore team then performs the user acceptance tests and validates the requirements and the defects.

Model 2: Complete Offshoring

The second model tries to provide as much colocation as possible by shipping the entire production team overseas. The only person left onshore is the customer. The customer sends features and the offshore team responds with fully tested working code. This scenario is arguably the most cost-effective among all.

Model 3: Mirrored Offshore Team

The third and most effective model for companies transitioning to Agile is the mirrored-design model. This model can be less cost-effective in the beginning, than Model 2, however it the initial investment pays itself out with less defects and shorter development cycles. In addition, the mirrored teams lay the groundwork for a smooth transition by gradually shifting responsibilities to the offshore team.

Agile 101: Automated Build Scripts (ABS) defined

What is ABS?

Automated Build Scripts (ABS) provide an ability to build software in an unattended and machine-independent environment. ABS are generally written and maintained by a special group of developers called configuration managers, or software integrators or just application builders. Some of the tools used to write ABS are Ant, NAnt, Maven, MSBuild, Visual Build Pro, and make.

Automated builds are one of the most important practices for any Agile team. They help developers see the effects of any change immediately and make it possible to deliver working software at the end of each iteration. The need for ABS is even more pronounced in distributed Agile teams because multiple people build simultaneously and in the absence of a common team room it is paramount that every change is properly integrated into the build. This is a fundamental attribute of a well designed automated build system since it helps keep the repository coherent and the code base buildable.

Other important features of good ABS are:

• Allow a system to be built on any machine while requiring minimal (properties) or no changes to the base scripts - in most Agile teams it is more than likely that teams will have different machine settings such as OS, language, IDE. The ABS should be machine- and environment-independent.
• ABS should be easy to plug-in to a Continuous Integration (CI) sever - this is especially important for distributed and offshore Agile teams as the time zone differences make it necessary to build continuously and notify the teams of any problems early. ABS written in Ant or Maven for instance, allow for automated and unattended execution, making them a perfect choice for CI.
• Ability to execute ABS with a simple shell command - both the architect and the developers should be able to trigger a full build with a simple shell command. The best ABS are designed to be simple and guiding the user through the steps so that even relatively non-technical team members such as business analysts and project managers are able to stand-up a system with minimal help from the rest of the team.
• Ability to define and manage project dependencies. These are both compile-time dependencies such as libraries as well-as deploy-time dependencies such database scripts and XML configuration files. In highly collaborative Agile projects the DB servers and XML configurations are likely to change often, so a good ABS should take care of the dependencies in a transparent manner.

User Experience in Agile

User Experience (UE) is probably one of the least understood fields in todays software creation community. Yet it is arguably the most important thing we humans look for in an application. One of the companies that understands UE is Google. That’s one of the reasons they are building a religious user base. Truly yours is a fan of their applications including Reader and Maps and although they seem to have blooped with Desktop Search, they are still topping my and many other users’ most likable applications lists. In fact it is not a coincidence that many of us have come to expect the next cool technology or cool app to come from no one else, but Google. So, how does Google do its magic? By understanding human behavior and creating human-friendly applications.

Agile methods are particularly well suited for applying good UE principle in your application and achieving Google-like UE designs. I’d like to list ten practical recommendations from my personal experiences that could help better understand where UE fits and how it can be used to add value to the application and make its users feel good (…yes, I feel good every time I use Reader).

Hire the right people

Start with recruiting the right talent for the job. Do not underestimate this step, it is both difficult to locate good people in this fields and also to tell the good from the bad. Look for people with social sciences backgrounds and experience working in fields such as cognitive psychology, interaction design, information systems design, semantic user interface design, context-based interactions. Shy away from engineers and graphic designers [ok, this is sooooo general, that perhaps is worthless] because we tend to be too analytical, systematic and predictable [read:boring]

Make the UE part of the Onsite Customer

Because UE is such an important task it goes hand in hand with the business requirements. Make the Product Manager (PM) and the UE expert share the responsibility of getting the right requirements to the User Interface (UI) team, the business analysts and the programmers. Let the UE participate in the story prioritization and iteration-end demos. Take their feedback seriously and treat is a real business requirement, equal to the ones coming from the PM.

Plan for UE

If you want users to love your software then take UE seriously. Plan for it. Allocate a few points each iteration for UE. Start with the lowest possible unit (point) and find-tune as you go. If your app is UE-centric, you may find yourself spending up to 10% of each iteration on UE. That’s quite normal and it shows you are dedicating enough time for the cool factor, besides the useful stuff. You will find yourself spending those points on clarifications, UE acceptance testing and reworking the UI based on valuable feedback from UE.

Automate the UE testing

UE testing is the activity of making sure that the users have a pleasant time using your application. So if humans and pleasure is involved how can an automated test help? Easy. Concentrate on parameters such as how much time it takes for a particular screen to render and capture that data. Fail the test if it is more than a threshold representing the “patience coefficient” of a user. Look for misaligned elements in your screen and fail the test when they happen. A common failure in AJAX-based systems is the connection and when that happens parts of the screen remain blank for a certain period of time. Try to automate as much as you can using tools such as Selenium and Watir. Make the automated UE testing part of your continuous integration loop and raise hell when it breaks just as you’d do if my JSP fails to compile.

Internal UE releases

When does UE start and end? It does not. It’s there all the time, from beginning until the end. One of the advantages of the Agile methods is that working software is produced at the end of each iteration, so make it a habit to release a working copy to the UE team to play with while the programmers are working on the next set of stories. This will give the UE team a chance to test the application in a more relaxed environment and therefore they can spend more time evaluating its usefulness than the limited demo at the end of each iteration.

Keep UE independent

They are the people who should be able to design and evaluate the software as an outsider, so keep the UE outside of the core team. If this means exclusion from the common team area, do it. Let them stay independent and give them enough time and space to evaluate your interim product on a regular basis. This independence will eventually manifest itself as great features, likable user experience and fresh new ideas. You can occasionally invite them to your parties, that is OK; we all need well-dressed people around us khaki-knowing-and-loving engineers.

UE Acceptance Testing

Make the UE experts the ones who decide whether your iteration was successful in the UE department or you missed the mark. Invite them to the iteration-end demo and let them evaluate the software based on independent evaluation criteria. Do not postpone UE rework as a result of the feedbacks and if possible tackle failed features in the next immediate iteration. Do not allow for the accumulation of UE debt.

UE mockups are not UI mockups

Make sure you and your team understand that what UE creates as UE mockups (wireframes) are not screenshots the future application. That is the UI team’s responsibility which starts by them getting the UE mock and working with the PM and UE to create a UI mockup. UE is said to create low-fidelity wireframes, sometimes drawn by hand, and the UI converts them into high-fidelity screen mockups. This is an important distinction as many team members seem confused about the various artifacts produced. Think of the UE mocks as the principles and the UI mockups as the prototypes that implement those principles.

UE and Architecture go hand in hand

UE works closely with the Agile architect. What will eventually become the domain model starts as an idea in the UE’s brains. UE and Architects should share brainstorming sessions and use whiteboard design, and loose notation [read:non-UML] to capture their artifacts.

Get creative

Just because you have not heard about it does not mean it’s not possible. Get creative and use other disciplines to interact with UE and allow them better means to share their non-technical ideas [and passions too]. For example import the concept of “storyboards” from the movie industry. Treat each screen of your software as a scene from a movie. Sketch your players, mark your cameras and lights and shout “Action”. See what happens. Map the paths a user can traverse to get in and out of the particular screen and try to imagine how he/she feels.

UML has been particularly difficult for non-engineers to understand. Use other, less-formal means to communicate with the creatives of the team. For example use mind maps whenever possible to draft user paths and behaviors. Use the mind maps to work on a domain model. The relaxed notation of the mind map makes it the ideal choice to put UE, PM, business analysts and architects on the same drawing board.

Agile Monkeys Gone Wild

One of my favorite readings on Agile is Barry Boehm and Richard Turner’s now classic work “Balancing Agility and Discipline” [1]. In fact I made it a habit to revisit the book every once in a while. The prelude section features an interesting story about the elephant and the monkey. The story has a happy ending where the trusted elephant and the agile monkey join forces to serve their villages better. That’s really cool I say, but what if the monkey does not want to play? In other words, what if your village’s agile monkey wants to stay on the wild side and continue doing what iit is doing? What can your village do?

Extreme Agilists can present a real risk to any company developing software in an Agile manner. Working with people set on a particular model and unwilling to bend for the greater good of the whole could be particularly challenging for teams in the process of transitioning to an Agile method.

From my experience there are at least two major drivers for an extreme Agilist behavior. The first is the failure to link technical and methodology objectives to the business objectives of the company. This scenario usually happens when the Agilist takes the team into a pre-cooked method or solution with little regard to the real objectives of the company, such as markets, competitors, product positioning and overall strategy.

The second and perhaps more important driver is the over-emphasis on Agile methods rather than fundamental software engineering practices. A typical example of the latter is the way upfront design and architecture is handled in certain Agile methods. It is easy to fall into the trap of assuming that very little or no upfront design is going to work, just because of an Agile method. The reality is that in most cases the architecture does not evolve on its own and it needs a certain babysitting from the architects and the team. There needs to be balance of upfront architecture plus evolutionary design to achieve a middle way of delivering solid architecture in the final product.

So, what can the village do if they have to deal with a rogue agile monkey? They can start by talking about the real purpose of the project and the team’s existence. They can make it clear that most companies in today’s world cannot afford to subscribe solely to any particular platform, technology, or methodology. All of these are tools in the company’s disposal to achieve its final objective. Companies today pick their tools based on their short and long term business vision and the toolset should reflect that.

Finally if nothing else works I recommend the village elders get together with the monkeys and tell them the story of Babel. Per Frederick P. Brooks Jr. [2] despite that the project had a clear vision, plenty of manpower, adequate technology, lots of materials and enough time, it still failed. The project was a big initiative for its time. In fact, according to the Genesis account, the tower of Babel was man’s second major engineering undertaking, after Noah’s ark. But Babel was the first engineering failure.

Babel failed in two respects—communication and organization. The man were unable to talk with each other; hence they could not coordinate. The lack of communication led to disputes, bad feelings, and group jealousies. Shortly the clans began to move apart, preferring isolation to wrangling.

The village elders should talk to the agile monkeys and try to persuade them to play with the elephant. The alternative would be isolation and that could ultimately lead to the failure of the project.

References:

[1] Balancing Agility and Discipline: A Guide for the Perplexed, Barry Boehm, Richard Turner - http://www.amazon.com/Balancing-Agility-Discipline-Guide-Perplexed/dp/0321186125

[2] Mythical Man-Month, The: Essays on Software Engineering, Anniversary Edition, Frederick P. Brooks Jr. - http://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959

Continuous Refactoring and ROI

“So, just to make sure I understand correctly, you’re saying you would like the entire development team for one month to work on cleaning technical issues and this has nothing to do with our latest set of requirements. So, we are not adding new features…, right?” asked the VP, sinking in the sea of technical people surrounding him. “Yes, that’s our plan…“, responded the Director of Development. “…I consulted with my team leads and we are all in agreement that adding more features to the system without cleaning the technical debt will contribute to the mounting cost of fixing defects and changing various parts of the system. We as a team feel embarrassed to say to customers that even some small changes will take weeks.” “Show me the ROI!” impatiently shouted the VP of Engineering, “…that’s the question I’ll get from upstairs. So what’s our answer?”. “If we are asking to spend a whole month allocating the entire team to cleaning this technical debt, then we’d better have the right answer to these two questions: first, why is this much debt accumulated anyway - why can’t we get it right the first time? And then, let’s assume I am able to buy you this time, is this going to solve all of our issues once and for all?“

This dialog is all too common in software development teams today. It reflects a feature-driven brainset where product management is anxious to get as many new features to the market as soon as [mostly] humanly possible. In this marathon of features, usually no time is allocated-or-left for architecting a proper solution. Similarly, after a release gets out no time is spared to clean-up leftovers.

Company executives fear of refactoring, because they think it has no positive effect on the marketability of the product. But it hurts the bottom line and so they develop severe fear for changes to the software that have no proven ROI, which I’ll simply call refactophobia [define: obsessive, persistent, unrealistic fear of an external object or situation, in this case of changes to a software system with no prospects of increase in revenue]. They like to think of it as a frivolous luxury; something the tech guys enjoy doing every once in a while. Plus, there are no new features and since customers do not pay for a beautiful code, what’s the point?

I’ll try to explain the point. Agile development promotes the practice of continuous refactoring, a practice which plans small refactoring into each iteration and therefore does not allow for the accumulation of any technical debt. There are significant cost savings to the teams following this practice. Let’s compare this approach to other alternatives.

Figure 1 - Software Products: Cost of Change vs. Time

Let’s take a look at Figure 1. The black line in the chart represents a traditional software product lifecycle. Complexity and code smells (badly designed or coded code) mounts as the product matures until it reaches a point of infeasibility. There are two major factors making the product infeasible: cost of change and cost of adding new features and cost of fixing defects. In fact, inability to maintain their product is one of the major reasons for many software companies’ demise. Linked to this is the falling behind competitors due to the lack of new features. The total cost of this strategy can be thought of as the area under the black line in the chart.

The pink line is a representative of the above conversation. Some companies manage to squeeze once-a-year technical debt refactoring marathons which are aimed to reduce complexity and clean-up the code. These marathons are costly, but effective. They solve pressing issues and let the team take a breath until the next string of features. This strategy, while better than no refactoring, is still costly as it lets the smells live too long in the system and mutate into more complex, nastier smells. The area under the pink line in the chart is smaller in size than the black line which is an indication of the cost savings of this method.

The last and most effective way of dealing with accumulated complexity in software products is continuous refactoring. Agile teams which practice short iterations are familiar with the concept of Iterative Incremental Design (IID). Another important practice is continuous refactoring, aka. fearless refactoring, aka. courageous refactoring. In the latter, teams plan ahead for the refactoring which will take place in each iteration by adding a few extra points for that iteration. In my experience the ratio of refactoring points to the whole has been in the 5-10% range. In reality it always goes the opposite to the planned, however the agile project plans have the magic of evening out after a certain period of time, so starting with the 5-10% figure may not be a bad idea. In the chart, continuous refactoring is represented by the green line, where as a result of small and incremental changes to the internals of the code with no effect on its external behaviors, the cost of change is kept under control.

“…so this chart looks sleek, but how am I going to explain that to the people in the board room?” …the confused VP was still mumbling …Well, there is not much any developer can do to fix that, except maybe reminding the managers of the simple values that most Agile developers are driven by: trust, courage, empowerment, accountability, respect, communication and simplicity.

On the Way to Agile Transparency: Climbing the Big Wall

You may have heard of Reinhold Messner. He is one of the world’s top climbers and he did not gain fame due to his personality (some argue he has lost his mind) or his PR skills, but because of his improvised, one-of-a-kind climbing style. He is the mountaineer who developed the agile climbing model. Ascend solo. Use no oxygen. Improvise routes. He wasn’t modest about his achievement: “As far as the public is concerned, since 1978, my sensational climbs - Everest without oxygen and Nanga Parbat solo - are unsurpassed” (1). Indeed. Agility is a good way to get there from here: whether you are climbing a mountain, or building software.

Agile development methodologies mean a lot of things to a lot of people these days, and most of these have to do with making life easier: as a business partner, you may want faster time to value and lower development costs. As a Director of Software Development, you may dream of a people-centric model, minimal methods and maximum collaboration (2).

Transparency is a major dynamic associated with agile development. At the roots of the people-centric model advocated by agile development is a philosophy of collaborative, non-punitive accountability that can be defined as transparency. Thanks to their social methodology, agile projects offer better transparency to clients, business partners and corporate decision makers. When broken down, this concept consists of management components such as individual responsibility, commitment, and accountability. Peter Drucker’s definition of individual responsibility in his book “A Functioning Society” aligns smoothly with the transparency model agile supporters believe in: “Responsibility is both external and internal. Externally it implies accountability to some person or body and accountability for specific performance. Internally it implies commitment. The Responsible Worker is not only a worker who is accountable for specific results but also who has authority to do whatever is necessary to produce these results and who, finally, is committed to these results as a personal achievement.” To paraphrase, Agile methods advocate transparency, not to be confused with lack of accountability. As Stephan Haeckel (1999), director of strategic studies at IBM’s Advanced Business Institute, has observed, “Organizational responsiveness comes from giving individuals and groups the freedom to behave in ad hoc ways to respond to unforeseen circumstances. For this reason, organizational roles are defined in terms of accountability for commitments to particular outcomes, rather than in terms of activities.” If we are going to build a hub organizational structure and manage outcomes rather than activities, then there needs to be a mechanism for teams within the project structure to manage their commitments to each other. Rather than have the project manager keep up with and manage all inter-team dependencies, the teams need to accept that responsibility themselves, just as development teams commit to customer teams on features.” Team leads should continue to exercise traditional management, leadership and coaching practices and perform reviews based on each employee’s contributions and performance. In the lack of such understanding and team protection an environment of fear and witch hunting may become unavoidable, and that’s a sure way to kill any well-intended process.

An organizational philosophy of transparency, when correctly implemented, can make an impact. Recently a friend shared with me a neat little solution they’ve developed to track Agile projects – they call it the Agile Project Management Dashboard. The system in place is ingeniously simple yet very powerful. Each developer, analyst or tester involved with a particular project is asked to enter the status of their tasks to a shared spreadsheet located on a network drive. Later on a Share Point process picks up the updated files and generates a neat project dashboard bundling important information such as:

• Color coded project health status
• Iteration and release estimates and current percent done
• Percentage deviations from original estimates
• Completed and remaining stories

Acknowledging that one of the key concepts of agile methods is transparency, my friend set a system where the root cause of every problem is exposed early and decision makers get a sense of the life cycle of each project an any given instant. In highly collaborative and transparent teams like this, it is important to take the time to educate top management about the implications of constant visibility. In the case of the dashboard, it is paramount to convey the right message to upper management: namely, that this dashboard is an internal status tracking tool and nothing more. Many organization fall into the trap of attaching accountability to simple tracking tools such as this one, and hence risk reducing the level of collaboration and individual responsibility. Some places go even further to tie employee performance reviews based on some metrics generated from transparent tools such as this. In an open dialog environment such as an Agile project, top management should be well aware that increased transparency should not come at the expense of holding individual team members accountable of things they may have little control over. Honesty seems to go a long way of establishing credibility and contributing to the transparency of any team. This was emphasized by my friend and also once expressed by management guru Jack Welch. Welch asserts that “…leaders establish trust with candor, transparency and credit. When leaders do not do this, they foster an environment of suspicion. No one really knows what is happening. As a result, the network of people within the organization begin to work at cross purposes from one another. It is important that critical candor be constructive. That it is not just to put someone in their place, but provide them a way to move to a different place. When you have that kind of communication stream going on, the sharing of credit becomes much more motivational. Honesty begets motivation to improve and perform at your best. The leader though must model this genuinely for it to be effective.”

It may be argued that agile is to software development what globalization is to Ford and GM – agile development promises to shrink traditional boundaries by advocating a social, people-centric model that derives its power from commitment and dedication. It seeks to enhance productivity by introducing faces into what used to be an anynomous code production chain. Transparency and trust coupled with governance and accountability increase visibility and encourage collaboration. When special care is taken to balance the newly increased visibility with non-punitive accountability, great results will be achieved. In the end, anyone can climb that big wall, if they chose the path of trust and transparency.

(1) Retrieved from: http://www.jerberyd.com/climbing/climbers/messner/. Messner is also cited at The Agile Web Design Manifesto, An Introduction, by Emily Chang and Max Kiesler.
(2) A definition of agile development can be found here

Hibernate: Return on Investment (ROI)

For technology experts and developers alike, the term Return on Investment (ROI) is a bit of a blurry subject. Yet, it is one of the most common themes during product evaluation and technology platform selection. It is mostly the management that seems to be interested in the projected ROI
for a specific product or service.

In the old days when corporations used to pay for software, the evaluation of a specific product with regards to its cost/benefit ratio and especially the cost to the enterprise was relatively easier. With the advancement of the open-source movement and the proliferation of free, proven, open-standard frameworks the traditional ROI analysis needed an update. More than the initial cost of an open-source product - common item in paid software, the overall cost or the Total Cost of Ownership (TCO) of the product needed to be placed under the spotlight.

Let me make one thing clear. For an enterprise psychologically unprepared to embrace open-source, there is very little any ROI analysis can do. Enterprises should begin with accepting the idea that open-source works, is “really-really” free, is here to stay, and that to get ahead they have to invest in it. After this acknowledgment, ROI analysis can be though as a just another standard procedure or cost projection tool.

In this paper, I will analyze the ROI analysis for open source projects and support my case with empirical numbers taken from a fairly large enterprise adopting Hibernate, an open-source Object/Relational (O/R) mapping tool. I’ll try to articulate a clear picture on the projected Return on Investment (ROI) both for the immediate and long-term gains.

I’ll present each item with its order of importance to the overall ROI. At
the end of the paper there will be a section with the risks associated to this adoption and some empirical numbers portraying a dollar amount estimate in a small pilot project.

Hibernate (www.hibernate.org)

Hibernate is a widely accepted, open-source Object/Relational mapping framework for the Java and J2EE environments. The Hibernate group has been recently acquired by Jboss, adding to the growing momentum of the technology.

Benefits

Price of Acquisition and licensing

Hibernate is distributed freely, under the LPGP license. The LGPL license is sufficiently flexible to allow the use of Hibernate in both open source and commercial projects.

Industry Momentum

Hibernate
is a widely used O/R Java solution. The momentum surrounding its stable technology is not just perceived, as there are two key events indicating as such:

• Gavin
  King (creator of Hibernate) was chosen to lead the Sun EJB Expert Group for EJB 3.0. This was due to proven techniques and advantages of the Hibernate solution. Undoubtedly, EJB 3.0 will favor Hibernate design and solutions.
• JBoss, arguably the most leveraged open-source J2EE application server, has emerged with Hibernate and now develops its technology.

EJB 3.0

The Sun EJB Expert Group for EJB 3.0 is dramatically changing the persistence aspects of EJB, both from a design and API perspective. This potentially could open up EJB usage to a larger set of business cases. Having a Hibernate knowledge base will ease the transition to future EJB technologies.

Isolated Initial Usage - Pilot Project

Using Hibernate on a pilot project will allow the enterprise and its architects to establish a standard set of practices with the technology. This could then be rolled out for the next release and beyond.

Ease of Use and Flexibility

Due to Hibernate’s declarative nature, it is quite easy to program to, as well as integrate with an existing J2EE application. Hibernate resides on the south side of any multi-layer application, close to the physical data layer. As such, it seamlessly blends into your application with minimal intrusion on your existing design patterns. As an example, one can continue to use DAOs or Entity EJBs with Hibernate in the same way you would use them without it. As risk mitigation strategy, Hibernate can be swapped out with minimal touch points on the design.

Hibernate adapts to the development process, no matter if a team starts with a design from scratch or works with a legacy database. Hibernate generates SQL, relieving developers from manual result set handling and object conversion. Hibernate also provides transparent persistence that
allows systems to use any database and to switch between databases.

Reduction in Application Code

The persistence of any domain component can be prototyped to gain insight on developer burden for the Hibernate technology. A particular DAO Java class for persisting a key domain object (i.e., User) can be reduced by over 50%.

Developer Productivity

Hibernate features a natural programming model that supports natural Object-Oriented idioms - inheritance, polymorphism, composition and the
Java collections framework. It can also run outside of an application server allowing you to expedite the code/test cycle and skip the deployment step.

Performance

Hibernate supports various caching managers. It is a fast performer, has a dual-layer cache architecture, and may be used in a cluster. It’s pure Java design also allows it to be used in any J2EE application server

Other Technical Advantages

• Support for Ultra-Fine-Grained Object Models - a rich variety of mappings for collections and dependent objects.
• No Build-Time Byte code Enhancement - there’s no extra code generation or byte code processing steps in your build procedure.
• Logical Query Language - Hibernate addresses both sides of the problem: not only how to get objects into the database but also how to get them out again.
• Transparent Persistence - no required interfaces or base classes for persistent classes. JavaBean style properties are persisted. No build-time source or byte code generation or processing for faster build procedures. Also includes support for extensive subset of Java APIs.
• Object-Oriented Query Language - powerful query language expressed in a familiar SQL-like language. Includes full support for polymorphic queries. Queries may also be expressed in the native SQL dialect of your database.
• Flexible Object/Relational Mapping -includes threedifferent O/R mapping strategies including 1) table-per-class hierarchy,  2) table-per-subclass (normalized mapping), and 3) table-per-concrete-class. Multiple objects to single row mapping allow for use of fine-grained object composition. Polymorphic associations including many-to-one, one-to-many, many-to-many,
  one-to-one, and ternary maps are also possible for all mapping strategies. Other features include Bi-directional Associations, Association Filtering, Collections of Basic Types, Indexed Collections, CompositeCollection Elements, and Lifecycle Objects.
• Simple API’s - includes Simple Core API forapplication code, ODMG3-style API as an alternative to the more expressive native API, Extension API to allow customization, and Metadata API for applications that require access to Hibernate’s persistence metadata.
• Automatic Primary Key Generation -multiplesynthetic key generation including built-in support for identify (auto-increment) columns, sequences, UUID algorithm, and HI/LO algorithm. Alsoincludes support for application assigned identifiers and composite keys.
• Object/Relational Mapping Definition -XML mapping documents that bridge the O/R impedance mismatch, define the O/R mapping, andgenerate (and export) database table / constraint creation scripts. Designed tobe edited by hand, the Hibernate O/R mapping DTDs offer a human-readableformat. Hibernate also offers XDoclet support.
• Caching - the Hibernate Dual-Layer CacheArchitecture (HDLCA) delivers thread safeness, non-blocking data access, session level cache, optional second-level cache, and optional query cache. Hibernate also works well when other applications have simultaneous access tothe database.
• Ultra-High Performance - includes lazy initiation, outer join fetching, batch fetching, support for optimistic locking with versioning and time stamping, no special database tables, and SQL generated at system initialization time. Hibernate was designed from the ground up to working in a cluster and delivers a highly scalable architecture. Hibernate consistently offers superior performance to straight JDBC coding. Transaction isolation is guaranteed by the manner in which Hibernate integrates with database and JTA transactions.
• J2EE Integration - Hibernate can be configured andmanaged via JMX. It can also be integrated with J2EE application servers including JBoss Application Server via JCA. Hibernate SessionFactory may be listed in JNDI. Session is serializable and can be kept by a stateful session bean or in a Servlet. Application Server data source may be used to provideJDBC connections. Transactions integrate with application server transactions via JTA.
• Ability to Run Outside J2EE Container - in order to accelerate the code/test cycle, you can run Hibernate outside of an application server and skip the deployment step. This is a popular feature among developers.
• Support for Wide Range of Databases -including Oracle, DB2, Sybase, MS SQL Server, PostgreSQL, MySQL, HypersonicSQL, MckoiSQL, SAP DB, Interbase, Pointbase, Progress, FrontBase, Ingres, Informix, andFirebird.
  

  Now the hard question: How much will I save with Hibernate?

  To answer this question you need to look at the Hibernate with two different lenses. First, for the short term the advantages are clear: less code, less defects, standard-compliance, and higher performance. On the long run, the advantages are even clearer. Future versions of J2EE EJB technologies will be based on Hibernate and companies utilizing it today will have a relative advantage in the field.

  Now, let me try to summarize the previously discussed items and quantify the gains in a dollar amount.

  I will start with a hypothetical team ready to use Hibernate in a pilot project.The team has 10 developers, none of which has any previous Hibernate experience. To support the developers the management brings in two consultants mainly to train and mentor developers during project inception, design and construction. The company decides to implement a second project with similar Level of Effort (LOE). In the second case there is no need for training andthus the company frees their consultants.

  		In-house persistence framework	Hibernate - First Project	Hibernate - Second Project	Average of two projects
  A	Number of developers	10	10	10	10
  B	Number of consultants	0	2	0	1
  C	Developer $ cost/hr	$65	$65	$65	$65
  D	Consultant $ cost/hr	0	$130	0	$65
  E	Number of persistent classes	100	100	100	100
  F	Lines of Code (LOC)/class (avg)	600	400	400	400
  G	LOE (persistent classes only)	3000 hrs	2000 hrs	2000 hrs	2000 hrs
  H	LOE (training)/developer	0 hrs	40 hrs	0	20
  	Total (G*C + A*B*H*D)	$195,000	$234,000	$130,000	$182,000

  Looking at these empirical numbers it is easy to spot the advantage of Hibernate. Just in two short projects the tangible costs are significantly lower than those ofthe in-house solution. There are also some intangible benefits in the above scenario, such as:

  • Recruiting personnel already familiar with an industry-standard tool is easier and cheaper
  • New recruits do not need to learn any proprietary persistence frameworks
  • Company value is increased because of its compliance to standards

  Conclusion

  No matter how long the list of technical features, there will always be costs associated with open-source project adoption. Just as with any new initiative, there will be some visible and some hidden costs which need special attention.

  For many open-source projects, the visible costs are minimal, while the chunk ofthe hidden cost is related to personnel training and retention. Open-source projects and Hibernate in particular seem to be well aware of this fact as theymake special efforts to provide their “customers” with quality documentation. Documentation […and the lack of it] has become an important cost factor affecting the mitigation of another cost factor - personnel training. Hence many open-source projects are benchmarked on the amount and quality of their documentation.

  Second important factor in the cost analysis is the amount of time spent chasing defects in the product. With paid software [in most cases] enterprises have the leverageof immediate and [mostly] reliable technical support. With the emerging new market of open-source support, this becomes less of a factor.

  Third, licensing issues and fear of litigation is still a big obstacle for the adoption of open-source by corporations. Company lawyers are the most likely group to oppose any open-source infusion into the enterprise from the simple fear of litigation. Licensing education is an important item on the open-source community’s agenda.

  Hibernate makes a great case of cost mitigation for most of these factors.

JMeter Script for Linux

I wrote a while ago that JMeter comes with several launch scripts for starting the testing tool in different environments and conditions. One of those scripts was jmeter-n.bat, designed for background execution. It was unusual that a proxy was not supported in the original version which prompted me to write one that does.

I was playing with JMeter last night on my Linux box when I realized that some shell scripts for Linux were not included in the default JMeter distribution (Ver 2.2). In fact the only shell script that I could find was jmeter, the default JMeter launch script which needs a Swing interface to run. Other scripts were simply missing and the one I needed was jmeter-n.bat. Clearly a jmeter-n.sh was needed. So I decided to write my own. Here is a simplified jmeter-n.sh only tested in Linux 2.6.

#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
 
#  ============================================
#  Non-GUI version of JMETER.BAT (WinNT/2K only)
#
#  Drop a JMX file on this batch script, and it
#  will run it in non-GUI mode, with a log file
#  formed from the input file name but with the
#  extension .jtl
#
#  Only the first parameter is used.
#
#  For other OSes, the script currently behaves
#  like jmeter.bat - patches welcome!
#
#  ============================================
 
JAVA_HOME=/usr/java/j2sdk1.4.2_03/
JAVA_EXEC=$JAVA_HOME/bin/java
JMETER_HOME=/usr/local/jakarta-jmeter-2.0.2
JMETER_CMD_LINE_ARGS=$1
 
HEAP='-Xms256m -Xmx256m'
NEW='-XX:NewSize=128m -XX:MaxNewSize=128m'
SURVIVOR='-XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=50%'
TENURING='-XX:MaxTenuringThreshold=2'
EVACUATION='-XX:MaxLiveObjectEvacuationRatio=20%'
RMIGC='-Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000'
PERM='-XX:PermSize=64m -XX:MaxPermSize=64m'
DEBUG='-verbose:gc -XX:+PrintTenuringDistribution'
ARGS="$HEAP $NEW $SURVIVOR $TENURING $EVACUATION $RMIGC $PERM $DEBUG"
$JAVA_EXEC $ARGS -jar $JMETER_HOME/bin/ApacheJMeter.jar -n -t $JMETER_CMD_LINE_ARGS

You can download the shell script from here. Do not forget to edit the environment variables (JAVA_HOME, JMETER_HOME) and $ chmod +x jmeter-n.sh before running the script.

JMeter Script with Proxy Support

JMeter comes with several launch scripts:

• jmeter.bat - GUI execution with proxy support:
  -H [proxy server hostname or ip address]
  -P [proxy server port]
  -u [username for proxy authentication - if required]
  -a [password for proxy authentication - if required]
• jmeter-n.bat - background mode with no proxy support
• jmeterw.bat - GUI mode using javaw with proxy support

For some reason proxy is NOT supported in the unattended script, jmeter-n.bat . An easy way to add proxy support to jmeter-n.bat is to change the launch statement in the following way:

Notice that various launch options such as memory, garbage collection and debug mode can also be changed from this file.