Amsterdamned

i’m back.

for the last two weeks, i’ve been staying in the lovely city of Amsterdam, working for a customer of my dutch colleagues. challenging, amusing, funny and resource-consuming, here’s a brief recap of my last 15 days.

first of all, thanks from the deep of my heart to Maurizio “daje forte” Mao Pillitu, for hosting me in his nice and comfortable home, just outside the city town. he’s been very kind and friendly, i hope i had in some way paid back with my italian-style cousine.

so, i’ve been working for Hippo, a young and energetic open-source company born around their CMS product: it’s a nice building down-town, just 15 walking minutes far from Dam square (yep, i loved walking through the city lanes after a full day of working). guys at Hippo are friendly and passionate, devoted to open-source; they also organize forge-fridays, sort of coding dojos with the focus on releasing working plugins (for Hippo CMS, of course) at the end of the afternoon.

Hippo CMS is having a lot of popularity among public institutions in the Netherlands, something my dutch colleagues have been working on hard also. but even if Hippo 7 is getting popular, there are still a lot of projects done with the older product version, Hippo 6. And that’s were my story begins.

i’ve been working for the municipality of Schijndel, a little dutch town, helping its IT management improve and automate meeting’s agenda and reports publishing. yeah, you heard it right: they record and publish (with a little delay, of course) audio and text content for every council’s meeting. being an italian citizen, all that transparency and devotion sounds strange, but is really laudable.

the first challenge i faced was, of course, translating all documentations from dutch to english, from analysis PDF to past emails with customer. i didn’t had everything clear at first, but thanks to double-checking with dutch colleagues i finally got it. (anyway, it’s funny almost every translation from dutch gets verb in the very last part of sentences. it really reminded my latin classes, while at college).

then i finally entered the dark tunnel: technology viscosity and indecent web of dependencies, also known as Maven 1. gosh, i really had to work hard to have a successful build on top of Java 1.4, Axis2 and Cocoon 2.1, which turned out to be classpath monkey-patching, using ant tasks, jelly scripts and maven postGoals. damn!

add lack of support from webservice’s developers and consultants, and the soup is ready to be served! in fact, i just had a working test environment (i mean, representative of customer’s one, with valid data) almost 3 days before the project scheduled end. that’s awesome, isn’t it? how did the hell i managed to get the work done?

applying what i later called the “abstract and adapt” strategy: understand the domain, abstract from implementation details, then adapt code when things get clearer. well, that’s the hexagonal architecture (but, you know, we like coining sexy names). so, i spent the whole first week coding the application logic decoupled from real system behaviour, which in fact was unknown. Agenda and its Repository, Content and Storage, Indexer and Importer, these are all roles i’ve been writing, test-driven, from day one. that’s not easy, and of course it’s risky; but it was the best i could do.

reading webservice specifications and WSDL, i could also guess how that slimmy layer should behave, but i really got it wrong at first! then, i had an ah-ah moment during the first weekend, and changed the webservice adapter in order reflect my new thoughts, without the need to modify domain logic so much (in fact, i also improved my domain knowledge). i changed unit tests, and added sort of spikes: tests with no assertions, just logging actual parsed responses, so that i could “see” with my eyes current webservice behaviour, at each test run.

and i was right! i clearly remember how shocking was reading in the console log some parsed data, when they finally were set up on test environment! you know, i was going for lunch, i ran all tests one more time, before locking down workstation, and i saw that: “parsed 6 agenda”, following by a so-nice full toString(). that was awesome, really: my tests told me setup was done before receiving a confirmation email by consultants, 30 minutes later!

than, i had my journey to Schijndel, to discuss deployment and testing on customer’s network. trip took 2 hours, i also had a 30 minutes stop in ’s-Hertogenbosch which i spent walking down-town, among nice gothic buildings and golden dragons.

it’s shocking how efficient dutch national transports website is, with its door-to-door journey planner, really. well, it’s a shame it’s not updated with temporarily moved bus stops, which could have saved me one hour in the late evening!

anyway, that’s it, a recap of techy stuff mixed with journey reports. thanks to the whole dutch office for the opportunity and drinks, looking forward to next works together!

Increment on the long term, iterate day by day

this is the transcription of the 10-minutes lighting talk i had last weekend during our company meeting in Amsterdam, which is based on the great article by Alistair Cockburn “Incremental means adding, iterative means reworking”. so, here it comes..

Incremental vs. iterative development

every project is big enough not to be completed just in one shot. so, we focus on delivering subsystems on a regular basis, e.g. one or two weeks. we face complexity with simplicity and feedback.

first strategy we can adopt is delivering complete subsystems, every week, splitting projects into small chunk of complete functionality, also known as user stories. we add features every other week: this is incremental development, the heart of every agile process.

in other words, we try to get it right in the first place, targeting quality for a final product.

another strategy is reaching the final product by revisions. we start from a rough prototype, and improve it by later adding details, till the final result. we do reworking: this is iterative development.

a great essay on the two strategies is Jeff Patton’s Monalisa example, as reported by Gojko Adzic on the waterfall trap.

back to the initial motto, here is my suggestion: even if you’re already doing incremental development, don’t forget to iterate. (it’s a shame the term “iterations” does not relate to iterative development, but to incremental development!)

why? because customer, not us, choose when to stop incrementing (adding) and iterating (reworking): you’ll be surprised by what a customer can consider good enough.

Time for an example

my team is currently working on an e-commerce website, and the customer asked for a complex checkout procedure: adding items to the shopping cart, mandatory login, shipping and billing addresses, payment details (e.g. credit card info), and finally a checkout with notification via email and to the reporting system.

during release and iteration planning, we first asked the customer to split the procedure into smaller user stories, so that we could understand and estimate with confidence. then, we started developing the most important ones; for example, no billing address nor reporting system notification were chosen.

this is incremental development.

one day was the turn of the shipping address user story: calculate shipping costs for each country (Italy has a fixed cost, other countries costs are taken from the database of our customer’s shipping company), displaying italian provinces when shipping to Italy, validating zip codes when in Italy (CAP). everything in a dynamic fashion.

having a lot of work to do, we decided to start with a minimal layout: a combo box for countries and a text displaying shipping costs for current country. it was far away from the final product, but gave us feedback that everything was working, from UI to database import script.

later, we decided to put two forms, for Italy and other countries, into the same page. that was an easy way for having a different layout and data for Italy. again, it would have probably not been the final product, but helped us focusing on designing UI and setting up data.

we finally moved on trying to understand how to dinamically change layout, based on current country. first idea was to use Javascript code client-side: something like having two hidden forms from which setting a third form’s innerHtml, and the like. we did a spike, but our result was not satisfying.

then, having our iteration going to end soon, we decided to propose to the customer a simpler way: an onchange event on the country combo box! dynamic, but server-side. it took one pomodoro for spiking a solution. then we just asked the customer, and after other two pomodoro of layout restructuring, it was done.

that was iterative development.

hope it does make sense to you: increment on the long term, iterate day by day!

ATFQ

after almost a full day of nervous coding and testing..
trying to simplify a complex scenario..
before making it more complex with new requirements..
i finally found out it was easier than i thought..
simply asking the customer what he wanted.

so, please, don’t forget to ask the f**ing question!

Gimme a name

i’m currently working on a Rails web application, i’m learning a lot about Ruby and ActiveRecord these days. after a few green bar on a quite complex search, yesterday i felt a little bit annoyed because i was not able to refactor enough the logic embeded in the query. that’s why i did a spike on an ActiveRecord feature for making queries clearer: named scopes.

typed rails spike for generating a new project from scratch, scaffolded User, then i started with this test case, simply looking for the youngest teen named ‘bob’:

class UserTest < ActiveSupport::TestCase

  def setup
    User.create!(:name => 'alice', :age => 11)
    User.create!(:name => 'mark', :age => 18)
    User.create!(:name => 'bob', :age => 12)
    User.create!(:name => 'bob', :age => 14)
  end

  test "spike" do
    found = User.youngest_teen_Bob
    assert_equal 12, found.age
  end
end

make it pass. the initial and obvious implementation was a mix of :first, where clauses and order by:

class User < ActiveRecord::Base

  def User.youngest_teen_Bob
    User.find :first,
      :conditions => ['name = ? AND age < ?', 'bob', 15],
      :order => 'age ASC'
  end
end

yep, the example is very simple. anyway, i don’t think the query is clear enough. with any other ORM (even a hand-written one), i would liked to use something nearer to the domain, moving a little bit away from SQL. so, first step could be separating condition on ‘name’ from the rest:

named_scope :with_name, lambda { |name|
  { :conditions => { :name => name } }
}

def User.youngest_teen_Bob
  User.with_name('bob').find(:first,
    :conditions => ['age < ?', 15],
    :order => 'age ASC'
  )
end

look at User.with_name('bob').find(..): we’re now using a named scope called with_name, that simply append to the current query a where clause on ‘name’. so far, so good, but i now want to go further. what do you think the find(..) selection is doing? in one sentece, “find the youngest teen”. ok, so let’s split it in two:

named_scope :teens,
  :conditions => [ 'age < ?', 15 ]

named_scope :with_name, lambda { |name|
  { :conditions => { :name => name } }
}

def User.youngest_teen_Bob
  User.teens.with_name('bob').find(:first, :order => 'age ASC')
end

great! another named_scope, teens, simplier because no parameter is passed. so, we’re quite done, actual search is User.teens.with_name('bob').find(..). again: what do you think the find(..) is doing? sure, looking for the youngest:

def self.youngest
  find(:first, :order => 'age ASC')
end

named_scope :teens,
  :conditions => [ 'age < ?', 15 ]

named_scope :with_name, lambda { |name|
  { :conditions => { :name => name } }
}

def User.youngest_teen_Bob
  teens.with_name('bob').youngest
end

done! youngest teen ‘bob’ is now implemented as teens.with_name('bob').youngest. nice, isn’t it?

here a few notes:

• teens.with_name(..) acts exactly as User.teens.with_name(..), no need to specify class, named scopes can be used from static methods
• youngest should be added at the end, because it’s invoking find. it’s silly: no way to use something like youngest.teen.with_name('bob'). if you’ve got any idea, drop me a line..

that’s all for today.

Keep monitoring

so, i finally managed to setup a wiki space for Cruise-monitor, thanks to my colleague Gustavo. i added a “five minutes tutorial”, to install and configure a monitor. Marco also sent me a shell script to “emulate” say on Ubuntu, wrapping festival TTS, so i added a “configure say on Ubuntu” page too.

Cruise-monitor is growing, feel free to contribute!

Say what you want

did you ever want to be notified about latest build status? think about having a script that monitors your build server via RSS feed, and notifies success or failures saying something like “build broken for project XYZ”. add a rubysh taste, and…

i’m happy to announce cruise-monitor has just been published! as README says:

“Cruise-monitor is, well, a monitor to CruiseControl build status, via RSS feed. It uses MacOS ’say’ command for notifications. So far, only CriseControl.rb is supported, but plans are to support CC and CC.NET as well”.

this is the first open-source project hosted on our company public servers that i’m involved in. it basically was born after a few broken builds for the project my team is currently working on: a Rails application built on a CruiseControl.rb continous integration server.

it happened a few times no one noticed a failing test or a missed svn add. we started using a feed reader playing sounds on each new build, but still it was not able to distinguish failure from success. that’s when we thought about using great MacOs say command.

it started as a spike on ruby, RSS and system exec, but it soon turned into a real project: a Rake build file; unit, integration and acceptance tests; README, LICENSE and TODO files.

so far i’m the only committer! i hope the community around will grow a little bit. i’m going to add details on our public confluence in the following days.. so, stay tuned!

Holy dayz

finally, it’s vacation time for me too!

i’m going to join spanish friends to a trip from Zaragoza to Andalusia. i’m leaving in a few hours, so.. see you in september!

About how pleasant a saturday can be

it all started at 6am.

yesterday i was in Bologna for the 4th Italian UGIAlt.NET conference. well, it was a very pleasant day: nice location, nice sessions, nice people, and finally a great social dinner. it also stopped raining just in time for open spaces to start, in the afteroon. so, we moved out in the garden just in front of the conference room and enjoyed fresh air sitting under the trees.

first of all, thanks to Roberto for the car journey and for being such a good friend.

so, here they are some notes i took during sessions.

Daniela started talking about designing User eXperience, focusing on prototyping as a way to let customer play and understand what he really needs, before producing functional mockups. it soon turned into a hot discussion about how to apply this approach to incremental software processes. as any upfront design activity, it’s clear how costly later changes could be. Daniela suggested the main cost is related to information architecture changes.
other notes:

• user-oriented process with early feedback
• UX scenarios really reminds user stories with acceptance criteria
• help a vocabulary emerge, like in Domain-Driven Design
• working on more wireframes in parallel requires continuos feedback from the team (daily), to achieve information integrity. as with standup meetings

then, it was Massimiliano turn talking about Mono, from a committer point of view! having shared also the journey with him, i can say for sure he’s a great computer science enthusiast. his session, very interesting, was about why the project began and it’s current goals. he then talked about current developing, such as for Moonlight (officially approved by Microsoft) and MonoDevelop. he shared some of the practices his team has: only a main stable trunk and branches for experimental features, strong health check with regression tests and a dedicated build farm for each target architecture.
here a few notes:

• two developing stacks: working on Windows vs. working on Linux. Novell chose the same strategy Microsoft did, reuse existing componenst, so suggested stack on Linux it’s a collection of wrappers: to GTK/Cocoa, to MySQL, and the like
• Java support powered by IKVM: just pay attention to how .NET AppDomains and Java Classloaders work
• something Mono gives you more: a micro-threading framework; a tool to bundle executables and framework together (mkbundle) on all supported platforms; an embedding API to achieve interoperability between Mono and native OS syscalls, cross-platform!
• licensing: LGPL for core (give back modifications to community), MIT for libriaries. Novell can do relicensing, mainly to big companies
• embedded Mono: highly optimized Mono packaging as an alternative to MS Compact Framework
• appliance tools: Novell provides a tool for making custom appliances as ISO or VMVare images
• application servers: that’s the most interesting one for me, looking for a valid open-source alternative to enterprise Java: mod_modo is a way, reusing Apache services as for security and balancing. Fast-CGI is another

after a rainy lunch, Simone started talking about ASP.NET MVC, a project to which he dedicated a soon-to-be-printed book. after a brief introduction to the framework and its goals (mainly, remove from WebForms the nasty postback abstraction!) he suggested some strategy to achieve controller-view data passing. examples were for a composed page where both summary (as for menus) and details are displayed. it was a very technical session, talking about “web/user-control” alternatives, and finally controllers and actions testability. funny enough, we noticed that while Microsoft finally got MVC (it’s 2009!) Java seems to go back to control-based web UIs like for JSF. i also won a nice t-shirt for a few interesting questions i asked!

then, the best session (in my view) started: Roberto talking about code metrics and refactoring on legacy code. he shared his current work experience on a huge Java codebase, with no test at all and big bosses asking for code quality numbers. he showed us metrics such as size, complexity, cohesion and coupling, and suggested no tool can say the whole truth until a contextual code review is performed. moreover, metrics should not be used to “grab” system state in a given time, but to evaluate system evolution during time.
he decided not to achieve quality in the immediate, because testing everything is costly and maybe a waste, but to use new functionalities on existing code to drive testing and refactoring. after having run tools on actual code and collected initial metrics, he spent a few week developing a mini testing framework to let characterization tests be written. those integration tests where written inspecting HTTP traffic with the system (a service in a SOA environment) talking to the database: he then prepared XML requests and expected responses and let tests compare expected with actual SOAP.
having integration tests, he started a mentoring activity with developers, studying OO priniciples, refactoring and unit testing. so, the team started writing unit tests for new and existing code, then refactoring. this was also driven by tools for code duplication detection. here some notes:

• the testing framework was easy to write: system already used Dependency Injection with Spring. he used testing facilities provided by Spring, such as automatic transaction rollback and auto-wiring
• he used a combination of open-source tools to export data from production database. thanks to the ORM (and no SQL in the application code), he imported data into an in-memory database, for testing
• documentation as a metric: this was asked by management, but he suggested code readability as an alternative
• refactoring smells and moves: turn comments into assertions; improper names suggest too many responsabilities, so extract code
• mentoring is more effective with examples taken from actual system code, instead of providing links to external articles and books
• start from testing and refactoring new code (then increase code coverage and remove duplication), instead of refactoring existing code (with no priority given)
• time spent testing and refactoring was soon payed off by time saved from manual deploy and verification. estimates were increased just by 20%. developers felt having much more time for developing

finally, i had the pleasure to drive an open-space in the garden with a dozen people or more. i started a discussion on team-building, taking notes on a flip-chart. my starting question was “how can we turn a workgroup into a team?”. people were interested, we talked about team values, dicipline and practices, and a comparison with sport teams such as in soccer. i just put ideas on this mind map:

conference organizers were very kind and decided to reward everyone proposed and drove open-spaces. my prize was a personal license for Balsamiq. Simone, Emanuele and Claudio: thank you guys!

finally, a special thank to all Bologna XUPG-ers for such a great dinner and for being so nice!

Flexible enough, but not too much

as usual, switching to a new project, i like to perform a code review on my own or while pairing, looking for un-emerged design or simply for duplications; that’s really the funniest thing for a design-maniac like me. sometime nice hints do come, sometimes don’t (probably because the design is good enough given actual goals).

so, last week i’ve moved to an existing codebase, and while pairing i noted down on paper some complexities i found on code; later, with all green bars, we tried to simplify a bit. what mainly come out was a revisited model-view-controller triad for a reporting system.

my preferite part was the formatting stuff: some nested complex loops became a composition of Formatters, each taking care of a small amount of rows (details, total) or columns (descriptions, calculations, and the like).

as we started a new story, i paid attention to how much code was reused: not as much as i supposed, and that was really a shame, even if code was clearer. today was our payoff.

we found a bug on a report. given a collection of aggregates, we expected to see details for first aggregate, then a summary for its total, followed by details for second aggregate, then second summary, and so on. instead, result was all details followed by all summaries. there was something wrong with how Formatters where composed. fortunately, that wiring was in one single point.

so, it turned out that it was just a matter of composing Formatters in a different way. from today journal:

we fixed a bug on rows order moving from a composite with two section formatters (details, summary) to a section formatter with a composite (details, summary). flexibility powered by OO.

day by day, i’m learning the difference between flexibility and generality (as Wake’s Refactoring Book helped me understand).

moreover, it now seems it’s also easier for us to estimate tasks, having clearer boundaries and reusing code. and, off course, knowing more about domain logic.

being honest, there’s a lot of noise due to Java generics! if only we could use .NET implementation..

Under the covers

i’ve grown up listening to my father’s Beatles tapes, that’s where my melodic soul come from, even if i later chose stronger and heavier tones.

yesterday i was a bit ill, and i spent a few hours surfing the web. i started looking for some great cover of one of my favourite Beatles song, “Eleanor Rigby”: it’s great both for music and lyrics. i suppose everyone know that song (“Ah, look at all the lonely people…”), even if it’s not one of the most famous.

i already knew the cover by Thrice, one of best one ever in my view, taken from the album “If We Could Only See Us Now”.

here they are some other good covers i’ve found on youtube:

• the Zoot, a 70’s australian band. even if i don’t really like glam rock, they re-arranged the song very well!
• Kim Weston, a sweet soul interpretation of the song
• Stanley Jordan, an incredible performance! he plays guitar like a piano
• London Symphonic Orchestra, quite similar to the original, due to the violins
• a Reggae mashup, with video

finally, i’ve found a lot of very bad just-scream-along metal versions. they probably don’t understand what a cover is: a re-arrangement, it’s not enough just to play with a different sound (as an ex-song-writer and musician, i’ve done the same mistake in the past). maybe, the only good one i’ve found is the old Realm cover, as performed in 1988.

it’s definetively a nice experiment to do, and it helped me forget my illness for a bit!