ATFQ

after almost a full day of nervous coding and testing..
trying to simplify a complex scenario..
before making it more complex with new requirements..
i finally found out it was easier than i thought..
simply asking the customer what he wanted.

so, please, don’t forget to ask the f**ing question!

Gimme a name

i’m currently working on a Rails web application, i’m learning a lot about Ruby and ActiveRecord these days. after a few green bar on a quite complex search, yesterday i felt a little bit annoyed because i was not able to refactor enough the logic embeded in the query. that’s why i did a spike on an ActiveRecord feature for making queries clearer: named scopes.

typed rails spike for generating a new project from scratch, scaffolded User, then i started with this test case, simply looking for the youngest teen named ‘bob’:

class UserTest < ActiveSupport::TestCase

  def setup
    User.create!(:name => 'alice', :age => 11)
    User.create!(:name => 'mark', :age => 18)
    User.create!(:name => 'bob', :age => 12)
    User.create!(:name => 'bob', :age => 14)
  end

  test "spike" do
    found = User.youngest_teen_Bob
    assert_equal 12, found.age
  end
end

make it pass. the initial and obvious implementation was a mix of :first, where clauses and order by:

class User < ActiveRecord::Base

  def User.youngest_teen_Bob
    User.find :first,
      :conditions => ['name = ? AND age < ?', 'bob', 15],
      :order => 'age ASC'
  end
end

yep, the example is very simple. anyway, i don’t think the query is clear enough. with any other ORM (even a hand-written one), i would liked to use something nearer to the domain, moving a little bit away from SQL. so, first step could be separating condition on ‘name’ from the rest:

named_scope :with_name, lambda { |name|
  { :conditions => { :name => name } }
}

def User.youngest_teen_Bob
  User.with_name('bob').find(:first,
    :conditions => ['age < ?', 15],
    :order => 'age ASC'
  )
end

look at User.with_name('bob').find(..): we’re now using a named scope called with_name, that simply append to the current query a where clause on ‘name’. so far, so good, but i now want to go further. what do you think the find(..) selection is doing? in one sentece, “find the youngest teen”. ok, so let’s split it in two:

named_scope :teens,
  :conditions => [ 'age < ?', 15 ]

named_scope :with_name, lambda { |name|
  { :conditions => { :name => name } }
}

def User.youngest_teen_Bob
  User.teens.with_name('bob').find(:first, :order => 'age ASC')
end

great! another named_scope, teens, simplier because no parameter is passed. so, we’re quite done, actual search is User.teens.with_name('bob').find(..). again: what do you think the find(..) is doing? sure, looking for the youngest:

def self.youngest
  find(:first, :order => 'age ASC')
end

named_scope :teens,
  :conditions => [ 'age < ?', 15 ]

named_scope :with_name, lambda { |name|
  { :conditions => { :name => name } }
}

def User.youngest_teen_Bob
  teens.with_name('bob').youngest
end

done! youngest teen ‘bob’ is now implemented as teens.with_name('bob').youngest. nice, isn’t it?

here a few notes:

• teens.with_name(..) acts exactly as User.teens.with_name(..), no need to specify class, named scopes can be used from static methods
• youngest should be added at the end, because it’s invoking find. it’s silly: no way to use something like youngest.teen.with_name('bob'). if you’ve got any idea, drop me a line..

that’s all for today.

Say what you want

did you ever want to be notified about latest build status? think about having a script that monitors your build server via RSS feed, and notifies success or failures saying something like “build broken for project XYZ”. add a rubysh taste, and…

i’m happy to announce cruise-monitor has just been published! as README says:

“Cruise-monitor is, well, a monitor to CruiseControl build status, via RSS feed. It uses MacOS ’say’ command for notifications. So far, only CriseControl.rb is supported, but plans are to support CC and CC.NET as well”.

this is the first open-source project hosted on our company public servers that i’m involved in. it basically was born after a few broken builds for the project my team is currently working on: a Rails application built on a CruiseControl.rb continous integration server.

it happened a few times no one noticed a failing test or a missed svn add. we started using a feed reader playing sounds on each new build, but still it was not able to distinguish failure from success. that’s when we thought about using great MacOs say command.

it started as a spike on ruby, RSS and system exec, but it soon turned into a real project: a Rake build file; unit, integration and acceptance tests; README, LICENSE and TODO files.

so far i’m the only committer! i hope the community around will grow a little bit. i’m going to add details on our public confluence in the following days.. so, stay tuned!

Make it clearer

good naming is first step for understanding domain you work into.

today i was looking at a session foused on learning from the master, in any creative field. Philippe pointed out some good advice from Beck’s “Smalltalk Best Practice Patterns” book; first of all is “good naming is not important, is crucial”. choosing a good vocabulary, focusing on roles, is step one to conquer complexity.

ok, but when is the right time to choose names?

Beck’s book, as Philippe states, teaches how to make good micro-decisions, every few minutes, while coding: that’s what Beck later called “implementation patterns”. yep, but coding is not the only right time to name things.

refactoring is another.

when i first studied Refactoring book what was clear to me was the distinction between refactoring as a noun (a modification which improve design but doesn’t change behaviour) and refactoring as a verb (the activity of modifying design). then, question is: when do you refactor, and what is the goal? let’s see..

last week, at least twice, my pair and i were in the middle of understanding which object was in charge of some new functionalities. when we found a “good” place where to start, first step was extracting a bunch of code into a new class to better focus on existing behaviour. moving code let us reflect on roles: does the “old” owner still have the same role? and what about the brand new class, which is its role? names should be choosen accordingly. refactoring can be applied before adding new functionalities, in order to make it simpler to add.

finally, a very good advice is to refactor code while doing code reviews. sometime, i found very useful to checkout a codebase from scratch, start looking at some code i don’t master and rename classes and methods as i understand roles. useless code and duplications emerge during this process. same strategy should be applied while pairing, after each green bar.

last advice for today: kids, don’t do this at home without an automatic tests suite!

Collections algorithms as infrastructure

this week i’ve performed my first public kata, at the local XPUG, here in Milan. meeting’s topic was “internal iterators and blocks”, very close to my heart. i just had the idea a few days ago, and the eXtreme guys approved my session as soon as i proposed. (this year seems promising, a lot of pratical sessions planned… stay tuned!)

the main idea for a (code) kata is to perform a short pratical exercise you feel confident with, almost with no braining; fingers should go on smoothly, like a dance. i chose to start from Matteo’s “birthday greetings” session on exagonal architecture, as coded by Milo and me during an XPUG meeting. i did the kata at home a few times and collected on paper some guidelines. then i printed out “brief instructions”, which i distributed to the audience before starting. the plan was a three short exercises kata.

first, i refactored existing code, splitting loops, extracting the InternalIterator / Predicate pair and then moving construction to facades like On an Being. second exercise was adding a new functionality, “send a kiss to female employees”; i drove implementation with unit tests, and focused on using just extracted iterator and predicate code. it finally looked like this:

public class BirthdayService { ...

  public void sendGreetings(OurDate ourDate) throws Exception {
    List employees = employeeFacade.loadEmployees();

    for (Employee employee : On.items(employees).collect(Being.bornOn(ourDate))) {
      sendGreetingTo(employee, happyBirthday(employee));
    }

    for (Employee employee : On.items(employees).collect(Being.female())) {
      sendGreetingTo(employee, "A kiss for you");
    }
  }
}

then, the mini collections library – On.items().collect(predicate) – was first modified to be generic (with Java’s Generics) and then improved with other functionalities such as:

• findFirst(predicate)
• reject(predicate)
• contains(predicate)
• apply(transformation)

final code consisted of 3 interfaces for blocks, one class for internal iterator, a Not predicate implementation and, finally, the On facade.

the leitmotif for the session was “you know, i don’t really like algorithms, so i don’t want to write them twice!”. that funny sentence was just trying to show how much more readable and understandable the code was without noisy algorithm details (like useless foreach, if/else, results.add(), if not null, etc). even if it was late evening, audience was really following me (and i have to admit, i was going a bit too fast!). they also asked me some good questions.

Gabriele pointed out that maybe findFirst() should provide a default value parameter, instead or returning a null reference. with Roberto we also discussed how difficult could be to implement returning a generic NullObject instance. Matteo went further on the idea of literal programming, suggesting something like:

employees.collect(bornToday).do(sendGreeting)

this would require iterator returning another iterator instead of a collection, or encapsulating every collection into a domain object.

something that really catched my attention was Gabriele suggesting moving predicates from Being facade into domain classes, with the pleasant side effect of removing useless getters on domain objects. that’s really a good idea! i’ll consider this for sure next time (kata was insipired by code from a project i no more work on).

then, Matteo finally asked me how many “collection traverse” algorithms i would have encapsulated into an iterator. i could not answer, i thougth it depends on business scenarios. but i just didn’t understand well the question: he was referring to the foreach loops, and that’s why he suggested me that probably i would not have written another foreach loop, they’re just MAP and REDUCE (transform and collect). nice hint, i should study some more functional programming; or even better, some more Smalltalk-Rubysh idioms! (yes, i’ve done a Google search for that. first rule to learn more: admit your ignorance)

here it comes the best part. before going home, i pushed for this methapor. exagonal architecture (and DIP) promotes to encapsulate low-level infrastructure behind an interface (a facade), to which high-level client code interacts with passing around domain objects; you just have to write and adapter once. for example, an EmployeeFacade interface and a DatabaseEmployeeAdapter – written just once – which takes Conditions to filter data searches; new business scenarios can be achieved providing different Conditions, without the need to modify infrastructure code.

in the same way, i consider collection manipulation an infrastructure issue: what really matters is behaviour to be run on collection items. that’s why i really like writing algorithms just once, and reuse them heavly providing specific behaviour for a given user story: concrete predicates, transformations and other blocks are the only domain code i have to write; iterators are hidden behind a (creation) facade, like On.

when i first thought about, this sounded like an epiphany, but i can see it’s not so easy to understand, or to agree with. let me just know what you think about.

ok, and finally, here is the code! tests firts:

@Test
public void sholdCollectItemsMathcingAPredicate() throws Exception {
  assertEquals(Arrays.asList(2, 6), On.items(Arrays.asList(2, 3, 6)).collect(new Even()));
}

@Test
public void shouldFindAnItemMathingAPredicate() throws Exception {
  assertEquals(2, On.items(Arrays.asList(1, 2, 4, 5)).findFirst(new Even()));
}

@Test
public void shouldReturnNullIfNotMatchingItemFound() throws Exception {
  assertNull(On.items(Arrays.asList(1, 3, 5)).findFirst(new Even()));
}

@Test
public void shouldRejectItemsNotMathingAPredicate() throws Exception {
  assertEquals(Arrays.asList(1, 5), On.items(Arrays.asList(1, 2, 5, 4)).reject(new Even()));
}

@Test
public void shouldDetectIfAMatchingItemIsFound() throws Exception {
  assertTrue(On.items(2, 5).contains(new Even()));
}

@Test
public void shouldApplyTransformationOnEacheItem() throws Exception {
  assertEquals(Arrays.asList(2, 4, 6), On.items(1, 2, 3).apply(new Doubler()));
}

public class Even implements Predicate<Integer>{
  public boolean evaluate(Integer item) {
    return item % 2 == 0;
  }
}

public class Doubler implements SimpleTransformation<Integer> {
  public Integer applyOn(Integer item) {
    return item * 2;
  }
}

then application code:

public interface Predicate<TYPE> {
  boolean evaluate(TYPE item);
}

public interface Transformation<FROM, TO> {
  public TO applyOn(FROM item);
}

public interface SimpleTransformation<TYPE> extends Transformation<TYPE, TYPE>{
}

public class InternalIterator<TYPE> {
  private final List<TYPE> items;

  public InternalIterator(List<TYPE> items) {
    this.items = items;
  }

  public List<TYPE> collect(Predicate<TYPE> predicate) {
    List<TYPE> result = new ArrayList<TYPE>();
    for (TYPE eachItem : items) {
      if (predicate.evaluate(eachItem)) {
        result.add(eachItem);
      }
    }
    return result;
  }

  public TYPE findFirst(Predicate<TYPE> predicate) {
    List<TYPE> result = collect(predicate);
    return result.isEmpty() ? null : result.get(0);
  }

  public List<TYPE> reject(Predicate<TYPE> predicate) {
    return collect(new Not<TYPE>(predicate));
  }

  public boolean contains(Predicate<TYPE> predicate) {
    return ! collect(predicate).isEmpty();
  }

  public <TO> List<TO> apply(Transformation<TYPE, TO> tranfromation) {
    List<TO> result = new ArrayList<TO>();
    for (TYPE eachItem : items) {
      result.add(tranfromation.applyOn(eachItem));
    }
    return result;
  }
}

public class Not<TYPE> implements Predicate<TYPE> {
  private final Predicate<TYPE> predicate;

  public Not(Predicate<TYPE> predicate) {
    this.predicate = predicate;
  }

  public boolean evaluate(TYPE item) {
    return ! predicate.evaluate(item);
  }
}

public class On {

  public static <TYPE> InternalIterator<TYPE> items(List<TYPE> items) {
    return new InternalIterator<TYPE>(items);
  }

  public static <TYPE> InternalIterator<TYPE> items(TYPE... items) {
    return On.items(Arrays.asList(items));
  }
}

Show all Java imports given a root package

here’s a simple shell script to collect all imports in Java source code files, given a root package.

find src -iname *.java \
| xargs grep -i "import another.project" \
| cut -d " " -f 2 | cut -d "." -f 1-3 \
| sort | uniq

results look like this:

another.project.domain
another.project.presentation
another.project.util

change “1-3″ to reflect how deep the filter should be.

thanx to Andrea for showing me cut command. i would have used my belowed write-only awk!

Chronicle of a (damn stupid!) bug

mom said “test everything that could possibly break”. now, how to define “everything”? well, the story goes like this…

last week, me and Antonio were pairing trying to find out what was causing a very strange behaviour in our codebase. a batch process, loading data from a file and populating an “ATM transactions” table, should then put duplicate records on a “duplicate ATM transactions” table. so far, so good, we worked on stories like this zillion times.

when our customer’s proxy finally run the batch on a PREPRODUCTION environment, something went wrong. she then collected a log file, from wich we started investingating where the problem could be. what a big surprise reading “unique constraint XYZ violated” for “duplicates” table! could be? we put no contraints at all on that table! even more, XYZ contrainst was declared on “transactions” table, not duplicates!

at first sight, there was nothing wrong in the codebase; we then tried to reproduce the bug on our DEVELOPMENT environment too, with no luck. so, we decided to log more debug infos, such as “executing INSERT INTO transactions” and “executing INSERT INTO duplicates”. test all, commit, deploy and go!

after another run, this time on TEST environment, bug was there again, but at least we got more log to read. can you see? yeah, debug info was “executing INSERT INTO duplicates”, followed by the silly “contraint violated” exception! gosh! how could that be?

support from customer’s internal technical staff didn’t help much, we just find out that there were “something” different from DEVELOPMENT and TEST/PRE environments. very strange. anyway, we compared genearted DDL for tables on both environments, but, again, nothing helpful.

then light came, thanks to our new team-mate Roberto, our little Oracle guru!

on our daily standup, we let the team know we were with no ideas, so Roberto proposed himself for a pairing session with me (the team is working on few projects, on which we turn pairs on a weekly basis). after reviewing together (useless) DDL scripts, he then started showing me some magic with Oracle “reflection” stuff: queries on metadata, such as tables, contraints, and so on.

i was talking with a team mate, who just asked me something, when i turned back on the desk, and found Roberto with a smily face saying “Jacopo, i know what’s wrong, but you’ve got to wait for our 5 minutes break!”. arghh! couldn’t really wait!

you know, the problem was in the only part we didn’t test: customer’s internal stored procedure invocations, for creating database object alias!

we use incremental SQL scripts to recreate from scratch the database structure: we just skip stored procedures for grants and alias, because they cannot be run in DEVELOPMENT environment. we delay that feedback to a manual run (a.k.a. a demo!) in TEST and PREPRODUCTION environments. and that’s exaclty what we had: feedback on our broken SQL scripts. one of the alias was wrong, duplicates table was pointing to transactions table! it was probably caused by a copy and paste from another script (and that’s the saddest part).

so, how to define “everything”? simple, everything!

Subversion snippet

after the 100th time.. how to do a complete Subversion revert?

svn revert . -R
svn st | awk -F "?" '{ print $2 }' | xargs rm -rf