CQ, Sling, Felix and my headaches

December 27 2009

these last six months have been incredibly full for me, i’ve learnt so many technologies and technical stuff: RubyOnRails web application development (and a bit of S3 cloud deploying), Hippo CMS 6 and Cocoon pipelines, and now Day CQ stack, which means JCR and Jackrabbit, Sling RESTful web framework, and OSGI bundles with Felix. oh my!

yep, i’m currently working for a big TLC italian company, developing their internal portal based on CQ5. i was completely new to content-repositories and web content management, but i got it quickly: it’s a different paradigm, data are modeled around resources, not around relations (as with relational databases).

btw, what i want to show is my journey with CQ stuff, and how our development approach has grown during the last weeks (and where it’s going). beware: there’s a lot of technical stuff (maven, Day CRX, Apache Sling, Apache Felix); i won’t explain everything in detail, so i’m referring to documentation and other blog posts.

so, first of all, start reading CQ tutorial on “How to Set Up the Development Environment with Eclipse”: please, spend almost one hour following all steps, even boring ones, like grabbing jars from CRX repository and putting them manually into local maven repository. in the end, you’ll have two projects (ui and core), one page with template (manually created and edited), executing a component as JSP script (imported through VLT), which uses “domain” logic provided by a plain old Java class (from core project). that’s a lot of stuff!

then, let’s enter the magical world of CQDE, a customized (old version of) Eclipse, which provide access to remote content (via webdav) from within an IDE, so that you can edit, compile and debug code as it was stored locally (but it isn’t). at first, it seems a lot better than VLT-ing from commandline; but soon you’ll miss it: versioning, and sharing code with others. even if it’s not clear in the tutorial, ignoring VLT specific files let Subversion version also content stored in src/main/content/jcr_root. that’s not always funny, like manually merging conflicts on XML files, but it’s really a lot better than blindly edit code with CQDE, with no way back! also, sometimes i’ve found much more easier editing pages as XML files than using WCM editor (CQ authoring tool).

ok, relax, take a deep breath, and think about what you’ve done so far. do you like it? are you comfortable with this? well, i wasn’t; i missed my IDE-based development, checking-in and out code, running automatic tests all the time. the good news is we can do better than this, the bad news is we’ll still miss something (so far, red/green bars for UI). to recap, we can choose from:

  1. remote coding and debugging, with CQDE: no “native” versioning, VLT can be use as a “bridge” to Subversion
  2. local coding, with any IDE (eg Eclipse): still can’t compile JSP files, VLT used to deploy UI code

next step is (well, i’m a bit afraid, but time has come)… deploy an OSGI bundle with maven, with both UI code and initial content to put on repository.

step one: compiling JSP files locally. ingredients: JARs as local maven dependencies and sling maven jspc plugin.

i could not find any public Day maven repository (and it makes sense, from a business point of view), but as the tutorial shows, everything we need is already available from CRX. so, it takes long, but referring to the /libs/xyz/install convention and doing searches via CRX explorer you can come up with something like this:

#!/bin/bash

function grabDependency(){
  JAR_URL=$1
  REPOSITORY_DIR=~/.m2/repository/$2
  JAR_FILE=$3

  wget --user=admin --password=admin $JAR_URL
  mkdir -p $REPOSITORY_DIR
  mv $JAR_FILE $REPOSITORY_DIR
}

cd /tmp; rm -rf deps; mkdir deps; cd deps

grabDependency \
  http://localhost:4502/crx/repository/crx.default/libs/commons/install/day-commons-jstl-1.1.2.jar \
  com/day/commons/day-commons-jstl/1.1.2 \
  day-commons-jstl-1.1.2.jar

# ... grab other jar files

then, let’s add JSPC plugin to the maven build chain, and CQ and Sling dependencies (see attached file with sample code). this is a simple example; you’ll probably need to override plugin’s sling jar dependencies with versions used by application code!

<plugin>
  <groupId>org.apache.sling</groupId>
  <artifactId>maven-jspc-plugin</artifactId>
  <configuration>
    <compilerSourceVM>1.5</compilerSourceVM>
    <compilerTargetVM>1.5</compilerTargetVM>
  </configuration>
  <executions>
    <execution>
      <id>compile-jsp</id>
      <goals>
        <goal>jspc</goal>
      </goals>
    </execution>
  </executions>
</plugin>

moving JSP code into src/main/scripts (under apps/myApp subfolder) should be enough to have maven build (mvn clean compile). just remember to grab global.jsp from CRX and put it under src/main/scripts/libs/wcm folder. Eclipse also will compile (regenerate project files with mvn eclipse:eclipse), but it needs another copy of global.jsp into /libs/wcm (i know, it’s silly; i’ll check this next time).

step two: packaging an OSGI bundle with UI code and content nodes. ingredients: Felix maven bundle plugin.

the key concept for me was understanding what to put into the bundle. i was used to have JSP files on CRX under /apps node, editing nodes properties such as jcr:primaryType (cq:Component, cq:Template and the like) and jcr:content. deploying application as OSGI bundle it’s slightly different: code is available as bundle resources (from the bundle itself), while only property nodes are copied from bundle to CRX repository, as initial content. this separation was not clear to me in the beginning, but it now makes sense (even if less duplication would be nice, for example in content structure).

so, we should create a bundle with:

  • included resources: all required resources (maven resources and src/main/scripts folder) to be later referred
  • bundle resources: .class and JSP files
  • initial content: node properties, as JSON files (i decided to put them into src/main/resources, under CQ-INF/initial-content subfolder)

more details are available on the Sling website and on this blog post.

so, let’s add Felix bundle plugin to maven (remember to declare project bundle packaging with <packaging>bundle</packaging>):

<plugin>
  <groupId>org.apache.felix</groupId>
  <artifactId>maven-bundle-plugin</artifactId>
  <version>1.4.3</version>
  <extensions>true</extensions>
  <configuration>
    <instructions>
      <Export-Package>
        com.day.cq5.myapp.*;version=${pom.version},
        org.apache.jsp.apps.*;version=${pom.version}
      </Export-Package>
      <Import-Package>*</Import-Package>
      <Private-Package></Private-Package>

      <!--
          included resources folders (to be later referred):
          maven resources and JSP files
      -->
      <Include-Resource>
        {maven-resources},
        src/main/scripts
      </Include-Resource>

      <!--
          resources available from within bundle
          (not available as CRX nodes):
          compiled .class files and JSP files.
      -->
      <Sling-Bundle-Resources>
        /apps/myApp,
        /var/classes!/org/apache/jsp/apps/myApp
      </Sling-Bundle-Resources>

      <!--
          content initially copied into CRX nodes:
          properties as JSON descriptors
      -->
      <Sling-Initial-Content>
        CQ-INF/initial-content/apps/myApp/; overwrite:=true; path:=/apps/myApp,
        CQ-INF/initial-content/content/sample/; overwrite:=true; path:=/content/sample
      </Sling-Initial-Content>
    </instructions>
  </configuration>
</plugin>

this should be enough to create a package with mvn clean pakage. we’re almost done..

step three: installing the bundle. ingredients: maven sling plugin.

with CQ there are two ways to install a bundle: put it under /apps/myApp/install folder or using the Felix console. i choose the latter, which turns out to be a plain POST request to the console URL. anyway, we can hook the maven build chain with the Sling plugin, this way:

<plugin>
  <groupId>org.apache.sling</groupId>
  <artifactId>maven-sling-plugin</artifactId>
  <version>2.0.4-incubator</version>
  <executions>
    <execution>
      <id>install-bundle</id>
      <goals>
        <goal>install</goal>
      </goals>
      <configuration>
        <slingUrl>http://localhost:4502/system/console/install</slingUrl>
        <user>admin</user>
        <password>admin</password>
      </configuration>
    </execution>
  </executions>
</plugin>

just type mvn install and we’re done.

that’s it. a lot of setups, expecially if, like me, you’re new to maven and OSGI. anyway, i’ve written this mainly for later reference and to share thoughts with colleagues. i’ve shown three approaches to develop with CQ, tested in my daily work on the last month. in my view, deploying OSGI bundles is the best one, so far; it’s a trade-off between ease of use while debugging (yep, no UI automatic tests yet) and development lifecycle (versioning, building, packaging). i hope to gather much more info next year, and probably something will be easier! next step will be setting up automatic tests for JSP files, using Koskela’s JspTest tool.

sample code is here: please, follow README and have fun.

well, happy new year to everyone!

Advertisements

9 Responses to “CQ, Sling, Felix and my headaches”


  1. […] CQ, Sling, Felix and my headaches […]

  2. paul ford Says:

    Nice article! Have you developed any custom CQ components? I’m assuming you have given the article above on osgi bundling. That being said is there a good resource you can recommend for a step-by-step approach to creating a component for a newbie?

    thanks so much

    paul

  3. Sarwar Bhuiyan Says:

    Hi Jacopo,

    Nice to see you did a write up on the pains of getting started on setting up development environments for CQ development. It has been a while since you started but you might be interested in or have found some other shortcuts in the meantime:

    1. You can turn your CQ5 installation into a maven repository itself. See http://dev.day.com/kb/content/wiki/kb/cq5/Development/HowToUseCQ5AsMavenRepository (just install the package and that’s it. Any dependencies you need you can check in Felix the group name and artifact id). All the jars are in the CQ installation anyway.

    2. If you want to use the Day maven repository instead, you can request a user id and password if you are a Day customer.

    3. Local editing on Eclipse, there is a way to compile JSPs so that your includes work. Since the UI project is web project (minus the WEB-INF folder), we can add the correct Eclipse natures and generate the eclipse project. In my UI project’s pom.xml, I set some maven-eclipse-plugin configuration like this:

    org.apache.maven.plugins
    maven-eclipse-plugin
    2.6

    2.4

    org.eclipse.wst.jsdt.core.jsNature
    org.eclipse.wst.common.project.facet.core.nature
    org.eclipse.wst.common.modulecore.ModuleCoreNature
    org.eclipse.jem.workbench.JavaEMFNature
    org.eclipse.jdt.core.javanature

    org.eclipse.wst.jsdt.core.javascriptValidator
    org.eclipse.jdt.core.javabuilder
    org.eclipse.wst.common.project.facet.core.builder
    org.eclipse.wst.validation.validationbuilder

    org.eclipse.jdt.launching.JRE_CONTAINER
    org.eclipse.jst.j2ee.internal.web.container

    apps/myapp/components

    bin/classes
    1.5

    The wtpversion config sets the right stuff in there so that eclipse looks in the right places for source files (jsps).

    Hope this helps you somewhat. I’ve found this quite useful.

    All the best.

    Sarwar

  4. Sarwar Bhuiyan Says:

    Wups, it’s stripped out all the xml markup. What’s the proper way to paste xml code into your comment?


  5. <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-eclipse-plugin</artifactId>
                <version>2.6</version>
                <configuration>
                     <additionalProjectFacets>
                        <jst.web>2.4</jst.web>
                    </additionalProjectFacets>
                    <additionalProjectnatures>
                    <projectnature>org.eclipse.wst.jsdt.core.jsNature</projectnature>
                   <projectnature>
                         org.eclipse.wst.common.project.facet.core.nature
                  </projectnature>
                               <projectnature>org.eclipse.wst.common.modulecore.ModuleCoreNature</projectnature>
                        <projectnature>org.eclipse.jem.workbench.JavaEMFNature</projectnature>
                        <projectnature>org.eclipse.jdt.core.javanature</projectnature>
                    </additionalProjectnatures>
                    <buildcommands>
                        <buildcommand>org.eclipse.wst.jsdt.core.javascriptValidator</buildcommand>
                        <buildcommand>org.eclipse.jdt.core.javabuilder</buildcommand>
                        <buildcommand>org.eclipse.wst.common.project.facet.core.builder</buildcommand>
                        <buildcommand>org.eclipse.wst.validation.validationbuilder</buildcommand>
    
                    </buildcommands>
                    <classpathContainers>
                        <classpathContainer>org.eclipse.jdt.launching.JRE_CONTAINER</classpathContainer>
                        <classpathContainer>org.eclipse.jst.j2ee.internal.web.container</classpathContainer>
                    </classpathContainers>
                    <sourceIncludes>
                        <sourceInclude>apps/myapp/components</sourceInclude>
                    </sourceIncludes>
                    <buildOutputDirectory>bin/classes</buildOutputDirectory>
                    <wtpversion>1.5</wtpversion>
    
                </configuration>
    </plugin>
    

    Sorry for pasting so much.

  6. Dan Says:

    Also, if you want to avoid the pain of VLT, you can try VaultClipse. VaultClipse allows you to import, export and merge content to/from your CQ repository similar to how you do so with Subversion.

    Visit: http://vaultclipse.sourceforge.net to check it out.

  7. Frank M Says:

    CQ and CRX artifacts are available in Adobe/Day Maven Repository here:

    Adobe Day
    http://repo.adobe.com/archiva/repository/day-central/


  8. Hello, just wanted to mention, I liked this post.
    It was funny. Keep on posting!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: