There’s more than one way to skin a cat

iterative and incremental development means delivering functionalities one at a time driven by some priority critera (e.g. business value). similar functionalities happen to be developed not all at the same time. so, design focus on supporting all the functionalities planned for a given iteration, and nothing more. key to succeed are simplicity, flexibility and a wide automatic test suite.

imagine a financial project where customer needs to process via nightly batch all monetary transactions received daily. he ask for supporting critical calculations first, and set at lower priority to automatic double check (it can be done manually in the meantime). in a plan of a few months, 3 or 4 different calculations are going to be developed.

but we start from the first one, and let the code (aka design) support the others later.

just after the first user story get signed, design consists of three main roles:

• a BatchCommand that collects items and process calculations, and then updates state on items (e.g. date processed)
• a Gateway to the database that knows criteria to filter items and knows how to select and save items from/to the database
• a Connector in charge of handling connection to the database

here’s a brief recap using CRC cards:

next iteration starts, and customer select another calculation to be developed, which just differs in selection criteria. then, the team has to find out the simplest way to support this scenario. well, a few exists, of course. in this post, i’m focusing on some design strategy we can use, comparing them.

we need to change Gateway selection criteria. someone says go for inheritance, and override criteria in a subclass. someone else says use composition, and move responsibility to change into another object. you choose, me too: composition!

so, we change Gateway to delegate a collaborator to gather selection criteria: let’s say, a CriteriaBuilder, even better as an interface. with our favourite IDE we perform an “extract method” refactoring followed by “move method”, moving selection criteria to a criteria builder. then, we add another builder for current user story. finally, it’s just a matter of wiring two Gateways with two CriteriaBuilders. design ends up being like this:

what’s wrong with this? well, nothing i would say, dependencies are well handled: low-level details (Gateway) depend on abstraction (CriteriaBuilder interface) and high-level policies (concrete CriteriaBuilders) don’t depend on anything, they provide that abstraction. in other words, we succesfully applied Dependency-Inversion Principle. so, again, what’s wrong?

we created two different gateways, that’s the matter, in my view. we stated Gateway is an infrastructure detail, it “knows how to select and save items from/to the database”. so, why do we need two different ones? has infrastructure changed? not at all.

in OO an object has an “outside” and an “inside”. GoF, in the Decorator pattern, to explain this simple rule used the metaphor of an object skin and guts:

“Changing the skin of an object versus changing its guts. We can think of a decorator as a skin over an object that changes its behavior. An alternative is to change the object’s guts. The Strategy pattern is a good example of a pattern for changing the guts.”

now, imagine we move selection criteria “outside” from Gateway, into another object which collaborates with the Gateway, but doesn’t hold a reference to it: it wll be sent as a parameter of the message “please, apply selection criteria on this gateway”. let’s say we name this new role a Selector. we end up having one Gateway and two Selectors, this way:

that’s it.

to recap, my suggestion to you while doing design: whenever you find an opportunity to simplify object responsibilities, moving some of them away, ask yourself “is this a domain or an infrastructure issue?”. if possible, avoid mixing the two: leave objects cohesive, and apply DIP. then, all other things being equal, avoid having more instances of a given infrastructue class, unless you really need to set different infrastructures (in this example, a gateway to filesystem or an in-memory one).

as Beck says, “Design is beneficially relating elements”.

Fakes aren’t stubs

yep, title resembles Fowler’s article on interaction vs. state based testing. that was one of my first readings when i approached TDD, and soon i fell in love with mock objects. things change, of course, but that’s another story…

very close to my heart, there are “test doubles”, see Meszaros work on this. last xMas i also gave a presentation on that (a webinar to my colleagues in Amsterdam and Rome), and i tried to make clearer how and when to use doubles in tests. but i still missed one point.

testing in isolation. well, i want to test a bunch of code without depending on other parts of the application. in OO systems, code is moduled into classes. using Responsibility-Driven Desing terminology, you need to isolate tested class from responsibilities it collaborates with, which usually form one or more role.

while doing TDD, isolation isn’t a goal, but quite a medium: to add the smallest behaviour possible, you need to isolate from whatever isn’t stricly related to the task at hand. indeed, you often discover collaborations while wrinting an automatic test: roles emerge.

mock objects help on that: as the mockist fathers say, “mock roles, not objects”. hand-written mocks or dynamic mock object libraries are the way to go, choose one. these days, i’m with the former by default, and for using tools on more complex scenarios.

isolation can also be reached without the need to programmatically set behaviour (which is the essence of mock objects), relying on pre-set stubs, which usually behave in a default way (do nothing on void methods, return some simple, fixed values otherwise). stubs are really useful to simply ignore collaborations. if you want to, stubs can also be programmed, overriding default values.

then, it comes the time when you need to test a bunch of classes: say, integration tests which touch system boundaries, or acceptance tests focusing on domain, which don’t want to go live. of course, they’re coarse-grainer than unit tests. in these scenarios, you simply can’t mock-out collaborations. even more, you probably don’t want to.

to verify something has worked as expected, some real beaviour is needed: say, storing data on file instead of a RDBMS, reading input from a local file-system instead of an online storage, and the like. one of the best examples i’ve recently read is Pryce’s work on TDD of Asynchronous Systems, where he effectively substitute a few pieces of an application with fake ones.

in other words, fake objects are complete and consistent implementation of a given role (i.e. an interface). you just won’t probably use them into production code, even if they would do their job. most common example is an in-memory repository: store, update and retrieve objects in a coherent way. Fowler’s definition is:

“fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).”

sure, “fake” is not a great name! to get fakes less depressed and let them gain self-esteem, here’s my campaign: consider them first-class citizens, and please, stop calling them fake when you’re really just looking for a simpler, poorer, more stupid stub!

It seams open

i can clearly remember when i first discussed DIP and OCP with others: it was two years ago, during my apprenticeship as an XPer. to me, it was nothing new, i already had studied all the principles that now come under the SOLID ancronym. but, probably, i hadn’t digested them enough: something that just came later with experience.

Dependency Inversion was for sure my favourite since then, for its multiple implications depending on what “high level” and “low level” mean; in my view, there are at least two meanings: abstraction (high level policies vs. low level details) and layering (close-to-user layer such as GUI vs. infrastructure layer). much more, i loved its love-hate with Dependency Injection (maybe more on this in a separate post).

Open-Closed is harder to understand, at first. how could you “change without modifying”? abstraction is the key! let modules depend on abstraction, then provide new implementations when behaviour has to change, without the need to modify existing code. in other words, always depend on abstraction: which, in the end, is DIP itself. OCP and DIP are such “yin and yang” in software design: they help achieve each other.

then, when i first discussed OCP with team-mates, i pushed for an analogy with Feather’s Seam Model. it’s discussed in “Working Effectively with Legacy Code” book: use seams to let legacy (which means untested) code be tested. to be fair, my analogy was not welcomed too much! i had to force my thesis a bit, and in the end not everybody was convinced.

two years later, it happened again! indeed, a few months ago we had a study group on OCP. i was in charge of preparing material to study on, and i chose a few corollary articles: first chapter from GoF Design Patterns book, which focuses on “design targeting interfaces”, and WELC chapter 4 “the Seam Model”. this time i was more convincing, and the analogy between OCP and the Seam Model became clear during our study session! and now, i want to tell you too.

after reading the bunch of articles i prepared, i asked a colleague to state in a few words what OCP was about. he said “change behaviour without modifying code”. great! then, i asked him again to state what the Seam Model was about, and he said “let code behave in a different way without modifying it”. well.. nothing left to say!

abstraction is the key, that’s true. but what about code which doesn’t follow OCP/DIP? it doesn’t depend on abstraction. we can modify it, refactoring, but we need an automatic test suite, in order to guarantee no behavioural change. and that’s exactly what Feather’s model is about: change code a little bit putting seams, to test it in isolation.

on the other side: what seams can you use to test OCP-compliant code? of course, already existing abstractions: you just have to change enabling points (in a test, usually in fixture setup).

in the report for our study session, on our internal wiki, we wrote:

We then discussed what OCP and Feather’s Seam Model have in common:

• they seem the same idea, applied to reach different goals
  • OCP: put abstractions to isolate from future source code changes
  • Seam: put abstractions to test applications without changing its source code
• to recap
  • closure/abstraction = seam
  • “main() routine”/factories = enabling point

to be precise, Feather’s model is about three different techniques, useful for testing legacy code written in any language, not just object-oriented ones. he talks about preprocessing seam, linking seam and object seam. so, the analogy with OCP is just between abstractions and object seams, even if sometime linking techniques are also used to achieve abstraction (such as reflection or some configuration-based IoC tool).

so, when i watched Misko’s clean-code-talks videos, i was surprised to hear him use the term “seam” while talking about DI and SOLID principles: he confirmed my analogy have sense!

How long…

it has been a long time since i had my heart beating sooo fast!

• A Speculative Fiction
• Mate Ka Moris Ukun Rasik An
• Purina Hall Of Fame
• Apparently I’m a pc Fascist + Dear Coach’s Corner
• Less Talk More Rock + Rio De San Atlanta, Manitoba

thanks to Propagandhi for an extremely emotional show last sunday in Bologna.

Make it clearer

good naming is first step for understanding domain you work into.

today i was looking at a session foused on learning from the master, in any creative field. Philippe pointed out some good advice from Beck’s “Smalltalk Best Practice Patterns” book; first of all is “good naming is not important, is crucial”. choosing a good vocabulary, focusing on roles, is step one to conquer complexity.

ok, but when is the right time to choose names?

Beck’s book, as Philippe states, teaches how to make good micro-decisions, every few minutes, while coding: that’s what Beck later called “implementation patterns”. yep, but coding is not the only right time to name things.

refactoring is another.

when i first studied Refactoring book what was clear to me was the distinction between refactoring as a noun (a modification which improve design but doesn’t change behaviour) and refactoring as a verb (the activity of modifying design). then, question is: when do you refactor, and what is the goal? let’s see..

last week, at least twice, my pair and i were in the middle of understanding which object was in charge of some new functionalities. when we found a “good” place where to start, first step was extracting a bunch of code into a new class to better focus on existing behaviour. moving code let us reflect on roles: does the “old” owner still have the same role? and what about the brand new class, which is its role? names should be choosen accordingly. refactoring can be applied before adding new functionalities, in order to make it simpler to add.

finally, a very good advice is to refactor code while doing code reviews. sometime, i found very useful to checkout a codebase from scratch, start looking at some code i don’t master and rename classes and methods as i understand roles. useless code and duplications emerge during this process. same strategy should be applied while pairing, after each green bar.

last advice for today: kids, don’t do this at home without an automatic tests suite!

Retrospective hints

i was thiking of posting some thoughts about last retrospective my team just had. funny enough, today i opened my feed reader and found this post by Jay Field on exactly the same topic: let the retrospective be driven by an external facilitator, and do a safety check before starting.

last friday we had a team retrospective. doing project ones would now be impossible, while we’re working on 3/4 projects and rotating pairs each day (more on this later). moreover, this time frame (last few weeks) has been very hard for us all; you know, the “rise and fall”: hard-to-folllow customers, projects coming and leaving, zero-velocity iterations, a few bugs.

yeah, but we also had a great surprise: we now have a new UK managing director! having Marco in Milan last week, we proposed him to drive our retrospective.

he chose to start the meeting with a “ballot”: write down on a note how likely you are talking to others, on a scale from 5 to 1. i never tried this strategy, even if Piero and me talked about during our session at Italian Agile Day. it was really interesting, numbers weren’t fully as expected. good: feedback gathered.

we used the starfish form, also known as “start-doing, stop-doing, keep-doing, more-of, less-of”. this time, after a rough clustering, we focused on “good and bad” for each cluster, and then proposed actions to support the good or to avoid the bad. then, we performed a quick vote and chose rule champions. (i’m the champion for “shared pomodoro”… let’s see…)

to recap, what went well:

• we asked an external person to drive our retrospective; of course, he knew what a retrospective is and how to drive it. but he really didn’t know what our issues were. nevertheless, he was able to understand when a bit more discussion was good and when to stop it
• everyboby knew what the feeling and trust were; so, core issues have been addressed, being both important to everyone and source of frustation for a few

now the hard part: on monday we’ll start adopting some very strict rule, everyone voted! as usual, we’ll try for one week at least, then we’ll reflect on data and feelings. let’s adapt!

Collections algorithms as infrastructure

this week i’ve performed my first public kata, at the local XPUG, here in Milan. meeting’s topic was “internal iterators and blocks”, very close to my heart. i just had the idea a few days ago, and the eXtreme guys approved my session as soon as i proposed. (this year seems promising, a lot of pratical sessions planned… stay tuned!)

the main idea for a (code) kata is to perform a short pratical exercise you feel confident with, almost with no braining; fingers should go on smoothly, like a dance. i chose to start from Matteo’s “birthday greetings” session on exagonal architecture, as coded by Milo and me during an XPUG meeting. i did the kata at home a few times and collected on paper some guidelines. then i printed out “brief instructions”, which i distributed to the audience before starting. the plan was a three short exercises kata.

first, i refactored existing code, splitting loops, extracting the InternalIterator / Predicate pair and then moving construction to facades like On an Being. second exercise was adding a new functionality, “send a kiss to female employees”; i drove implementation with unit tests, and focused on using just extracted iterator and predicate code. it finally looked like this:

public class BirthdayService { ...

  public void sendGreetings(OurDate ourDate) throws Exception {
    List employees = employeeFacade.loadEmployees();

    for (Employee employee : On.items(employees).collect(Being.bornOn(ourDate))) {
      sendGreetingTo(employee, happyBirthday(employee));
    }

    for (Employee employee : On.items(employees).collect(Being.female())) {
      sendGreetingTo(employee, "A kiss for you");
    }
  }
}

then, the mini collections library – On.items().collect(predicate) – was first modified to be generic (with Java’s Generics) and then improved with other functionalities such as:

• findFirst(predicate)
• reject(predicate)
• contains(predicate)
• apply(transformation)

final code consisted of 3 interfaces for blocks, one class for internal iterator, a Not predicate implementation and, finally, the On facade.

the leitmotif for the session was “you know, i don’t really like algorithms, so i don’t want to write them twice!”. that funny sentence was just trying to show how much more readable and understandable the code was without noisy algorithm details (like useless foreach, if/else, results.add(), if not null, etc). even if it was late evening, audience was really following me (and i have to admit, i was going a bit too fast!). they also asked me some good questions.

Gabriele pointed out that maybe findFirst() should provide a default value parameter, instead or returning a null reference. with Roberto we also discussed how difficult could be to implement returning a generic NullObject instance. Matteo went further on the idea of literal programming, suggesting something like:

employees.collect(bornToday).do(sendGreeting)

this would require iterator returning another iterator instead of a collection, or encapsulating every collection into a domain object.

something that really catched my attention was Gabriele suggesting moving predicates from Being facade into domain classes, with the pleasant side effect of removing useless getters on domain objects. that’s really a good idea! i’ll consider this for sure next time (kata was insipired by code from a project i no more work on).

then, Matteo finally asked me how many “collection traverse” algorithms i would have encapsulated into an iterator. i could not answer, i thougth it depends on business scenarios. but i just didn’t understand well the question: he was referring to the foreach loops, and that’s why he suggested me that probably i would not have written another foreach loop, they’re just MAP and REDUCE (transform and collect). nice hint, i should study some more functional programming; or even better, some more Smalltalk-Rubysh idioms! (yes, i’ve done a Google search for that. first rule to learn more: admit your ignorance)

here it comes the best part. before going home, i pushed for this methapor. exagonal architecture (and DIP) promotes to encapsulate low-level infrastructure behind an interface (a facade), to which high-level client code interacts with passing around domain objects; you just have to write and adapter once. for example, an EmployeeFacade interface and a DatabaseEmployeeAdapter – written just once – which takes Conditions to filter data searches; new business scenarios can be achieved providing different Conditions, without the need to modify infrastructure code.

in the same way, i consider collection manipulation an infrastructure issue: what really matters is behaviour to be run on collection items. that’s why i really like writing algorithms just once, and reuse them heavly providing specific behaviour for a given user story: concrete predicates, transformations and other blocks are the only domain code i have to write; iterators are hidden behind a (creation) facade, like On.

when i first thought about, this sounded like an epiphany, but i can see it’s not so easy to understand, or to agree with. let me just know what you think about.

ok, and finally, here is the code! tests firts:

@Test
public void sholdCollectItemsMathcingAPredicate() throws Exception {
  assertEquals(Arrays.asList(2, 6), On.items(Arrays.asList(2, 3, 6)).collect(new Even()));
}

@Test
public void shouldFindAnItemMathingAPredicate() throws Exception {
  assertEquals(2, On.items(Arrays.asList(1, 2, 4, 5)).findFirst(new Even()));
}

@Test
public void shouldReturnNullIfNotMatchingItemFound() throws Exception {
  assertNull(On.items(Arrays.asList(1, 3, 5)).findFirst(new Even()));
}

@Test
public void shouldRejectItemsNotMathingAPredicate() throws Exception {
  assertEquals(Arrays.asList(1, 5), On.items(Arrays.asList(1, 2, 5, 4)).reject(new Even()));
}

@Test
public void shouldDetectIfAMatchingItemIsFound() throws Exception {
  assertTrue(On.items(2, 5).contains(new Even()));
}

@Test
public void shouldApplyTransformationOnEacheItem() throws Exception {
  assertEquals(Arrays.asList(2, 4, 6), On.items(1, 2, 3).apply(new Doubler()));
}

public class Even implements Predicate<Integer>{
  public boolean evaluate(Integer item) {
    return item % 2 == 0;
  }
}

public class Doubler implements SimpleTransformation<Integer> {
  public Integer applyOn(Integer item) {
    return item * 2;
  }
}

then application code:

public interface Predicate<TYPE> {
  boolean evaluate(TYPE item);
}

public interface Transformation<FROM, TO> {
  public TO applyOn(FROM item);
}

public interface SimpleTransformation<TYPE> extends Transformation<TYPE, TYPE>{
}

public class InternalIterator<TYPE> {
  private final List<TYPE> items;

  public InternalIterator(List<TYPE> items) {
    this.items = items;
  }

  public List<TYPE> collect(Predicate<TYPE> predicate) {
    List<TYPE> result = new ArrayList<TYPE>();
    for (TYPE eachItem : items) {
      if (predicate.evaluate(eachItem)) {
        result.add(eachItem);
      }
    }
    return result;
  }

  public TYPE findFirst(Predicate<TYPE> predicate) {
    List<TYPE> result = collect(predicate);
    return result.isEmpty() ? null : result.get(0);
  }

  public List<TYPE> reject(Predicate<TYPE> predicate) {
    return collect(new Not<TYPE>(predicate));
  }

  public boolean contains(Predicate<TYPE> predicate) {
    return ! collect(predicate).isEmpty();
  }

  public <TO> List<TO> apply(Transformation<TYPE, TO> tranfromation) {
    List<TO> result = new ArrayList<TO>();
    for (TYPE eachItem : items) {
      result.add(tranfromation.applyOn(eachItem));
    }
    return result;
  }
}

public class Not<TYPE> implements Predicate<TYPE> {
  private final Predicate<TYPE> predicate;

  public Not(Predicate<TYPE> predicate) {
    this.predicate = predicate;
  }

  public boolean evaluate(TYPE item) {
    return ! predicate.evaluate(item);
  }
}

public class On {

  public static <TYPE> InternalIterator<TYPE> items(List<TYPE> items) {
    return new InternalIterator<TYPE>(items);
  }

  public static <TYPE> InternalIterator<TYPE> items(TYPE... items) {
    return On.items(Arrays.asList(items));
  }
}

Show all Java imports given a root package

here’s a simple shell script to collect all imports in Java source code files, given a root package.

find src -iname *.java \
| xargs grep -i "import another.project" \
| cut -d " " -f 2 | cut -d "." -f 1-3 \
| sort | uniq

results look like this:

another.project.domain
another.project.presentation
another.project.util

change “1-3″ to reflect how deep the filter should be.

thanx to Andrea for showing me cut command. i would have used my belowed write-only awk!

Chronicle of a (damn stupid!) bug

mom said “test everything that could possibly break”. now, how to define “everything”? well, the story goes like this…

last week, me and Antonio were pairing trying to find out what was causing a very strange behaviour in our codebase. a batch process, loading data from a file and populating an “ATM transactions” table, should then put duplicate records on a “duplicate ATM transactions” table. so far, so good, we worked on stories like this zillion times.

when our customer’s proxy finally run the batch on a PREPRODUCTION environment, something went wrong. she then collected a log file, from wich we started investingating where the problem could be. what a big surprise reading “unique constraint XYZ violated” for “duplicates” table! could be? we put no contraints at all on that table! even more, XYZ contrainst was declared on “transactions” table, not duplicates!

at first sight, there was nothing wrong in the codebase; we then tried to reproduce the bug on our DEVELOPMENT environment too, with no luck. so, we decided to log more debug infos, such as “executing INSERT INTO transactions” and “executing INSERT INTO duplicates”. test all, commit, deploy and go!

after another run, this time on TEST environment, bug was there again, but at least we got more log to read. can you see? yeah, debug info was “executing INSERT INTO duplicates”, followed by the silly “contraint violated” exception! gosh! how could that be?

support from customer’s internal technical staff didn’t help much, we just find out that there were “something” different from DEVELOPMENT and TEST/PRE environments. very strange. anyway, we compared genearted DDL for tables on both environments, but, again, nothing helpful.

then light came, thanks to our new team-mate Roberto, our little Oracle guru!

on our daily standup, we let the team know we were with no ideas, so Roberto proposed himself for a pairing session with me (the team is working on few projects, on which we turn pairs on a weekly basis). after reviewing together (useless) DDL scripts, he then started showing me some magic with Oracle “reflection” stuff: queries on metadata, such as tables, contraints, and so on.

i was talking with a team mate, who just asked me something, when i turned back on the desk, and found Roberto with a smily face saying “Jacopo, i know what’s wrong, but you’ve got to wait for our 5 minutes break!”. arghh! couldn’t really wait!

you know, the problem was in the only part we didn’t test: customer’s internal stored procedure invocations, for creating database object alias!

we use incremental SQL scripts to recreate from scratch the database structure: we just skip stored procedures for grants and alias, because they cannot be run in DEVELOPMENT environment. we delay that feedback to a manual run (a.k.a. a demo!) in TEST and PREPRODUCTION environments. and that’s exaclty what we had: feedback on our broken SQL scripts. one of the alias was wrong, duplicates table was pointing to transactions table! it was probably caused by a copy and paste from another script (and that’s the saddest part).

so, how to define “everything”? simple, everything!

Working software

.. over comprehensive documentation.

yes, that’s one of the very clear sentences in the Agile Manifesto. but there’s more.

last tuesday i attended a presentation on Agile and XP Gabriele gave at the local Linux User Group in Cremona. the session was very good, people interested, a lot of discussion, me too, bringing my last two years experience. but please, always trust a geek saying “go on as long as you like, we can stay here ’till late night”: he’s serious!

Gabriele is a great speaker, i had never seen any of his presentations before. amazingly, in three hours and half, he just scratched the surface of common topics such as testing and refactoring, instead focusing on planning and (re)estimating.

i learned a lot too. mainly, my attention was caught by this sentence: “our main goal is not to develope software satisfying business requirements, our goal is to develope software our customer can gather value from”. in my view, this is the essence of any Agile development process, and, in fact, is also stated as the first principle in the manifesto too: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software” (emphasis mine).

i spent a lot of time studying, learning, using and adjusting user stories acceptance tools, and helping developers understand how much “do the thing right” and “do the right thing” are different. but, even then, having all the acceptance criteria clear stated, is our job to verify them? do our customer pay us to do whatever he ask to?

definitively not. we get money to deliver software someone can make money with, or do something less costly with. or maybe, software doing something not feasible doing by hand. choose one. anyway, our job is to let our customer achieve his goals using our product.

for me, that’s really a context shift, and a daily challenge. i’m always focused on the internals of a system’s design, it’s architecture, how pieces work together. i live in an iperuranio where objects talk each other; where ideas born, live and evolve, and finally get real as working software. i’ve never paid much attention to what software is used for, i’ve always considered it a detail.

what a mistake!

from now on, contract i’ve signed as a developer is:

i’m allowed to imagine, design and construct the best (and simplest) system i can think of, as long as my goal is to deliver value to the customer.

working software over working acceptance tests!