Another night of coding, another update on the Discobot project! For those unfamiliar with the Discobot project: it’s the project name for running Groovy on Android. We (Marcin Erdmann and Erik Pragt) are working towards an easy to use framework to run the newest version of Groovy on the newest version of Android. Currently, we’re not there yet, but we will demonstrate our progress at the next Groovy and Grails Exchange in London.

A small update though on our current progress, since it has been a while since our last update, and we’ve gotten a few requests to update you all. To start with: the project is still very alive, and we are working hard to create a streamlined environment for developing Groovy Android applications, including Gradle support, with some tasks to easily deploy your Groovy application on the Android emulator.

The last couple of weeks we’ve been struggling to work with the dynamics provided by Groovy. Groovy works a bit different compared to the standard Java runtime in Android. A small example:

class B {
  def pub() {'a'}
} 

def x = [pub: {'b'}] as B

This code dynamically creates a class at runtime. We have to make the Android environment aware of these classes, which is done by the DiscobotClassloader. This is quite tricky to accomplish, but the good news is, that after some days and nights of coding, it works now. Basic Groovy code, like

[1,2,3].each { println it }

was already working for quite some time, but now we support a great deal of the features provided by Groovy already at the moment. Even though we haven’t measured yet how far we are exactly in terms of features, but we’re on the right way.

What we’ll do now though, is to create an Android application which will run all Groovy unit tests. We’ve done it before, and around 30% of the unit tests worked on the Android emulator. Now, with the support for dynamically generating classes, which was one of the biggest problem, we should get far above this 30%. So, please bear with us and we’ll keep update you on the progress we’ll make. In the meantime, you can follow the progress we’re making in our Git repository!

After some hectic times at work caused by very tight deadlines and me getting married in the beginning of September we finally managed with Erik to get back to working on Discobot last week.

Some of you have probably already noticed that we have a talk on Discobot scheduled at Groovy & Grails Exchange 2011 in London which takes place in early December. Although the progress is good there is still quite a lot of work to be done on Discobot and we decided that probably one evening a week will not be enough. On the other hand knowing about the talk gives us an even stronger motivation to succeed!

Last week Erik continued his work on getting all Groovy tests running in the emulator. We found out that ‘on the fly generated’ classes which are created by Groovy when you for example cast a map of closure into a class using ‘as’ keyword cannot be loaded by the classloader. The reason is simple – classloader looks for class files but it should be looking for dex files. This is our main concern at the moment as it causes exceptions all over the place and breaks anything more complex than a ‘hello world’. From the code written by Hjalmar we’ve noticed that he also encountered the problem and we’re trying to apply his solution to see if it will work. So, it looks like there’s a lot of class loading fun ahead of us!

We’ve also got a response from guys behind Gradle plugin for Android about our pull request. They had several small remarks which I managed include in my second pull request. From what they said it looks like they should accept it in the coming days.

I’ve also looked for options regarding a repository in which to put our artefacts (Gradle plugin and the modyfied version of Groovy). I’ve found a nice blog about creating a a github based Maven repository and using it with Gradle. It seems easy and straightforward so probably we will go for this solution for now.

There might be different reasons for setting a browser window size for your Geb tests. For us it was simply the fact that some floating elements where overlapping with other elements rendering them unclickable when the browser window was set to occupy only the half of the screen and that’s the default size of a Firefox window started by Webdriver.

There were two requirements for the solution – it had to work on every machine the tests were run without any configuration (so we had to discard all of the solutions involving changing the Firefox profile) and also the window had to be maximized for each of our tests. Unfortunately WebDriver doesn’t allow to easily control browser window size, at least as long as this issue isn’t solved. Therefore we had to revert to resizing the window via Javascript which we don’t like, but it seems to be the only solution for now. Thanks to putting the code into our GebConfig.groovy we got the window maximized for every test. Following is the driver closure we needed to put in our Geb config:

driver = {
    def driver = new FirefoxDriver()
    driver.executeScript("window.resizeTo(screen.width, screen.height)")
    return driver
}

Note that you can use a similar technique to not only maximize but also set the window to a certain size and to do it on a per test basis if you put the resizing code into your test instead of the GebConfig.groovy.

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In our development environment, we use the complete Atlassian suite. We (or actually I) choose this environment because the products integrates so well together, but also because it integrates good with other products, like SpringSource Tool Suite, or, in our case, IntelliJ IDEA. And finally, the Atlassian suite has excellent support for building Grails projects, which, at the moment, is all we do. Simply put, we choose Atlassian because it rocks!

To integrate even better with the Atlassian Build Server (Bamboo), I’ve created a small script which will update the version number of the application with the Build Number. This way, we always know which build is running where. To do so, I created the following script.

includeTargets << grailsScript("_GrailsEvents")

target('default': "Sets the current application version") {
    def buildNumberFile = new File('build-number.txt')
    if (buildNumberFile.exists()) {
        def props = new Properties()
        buildNumberFile.withInputStream { stream ->
            props.load(stream)
        }

        def buildNumber = props["build.number"]

        metadata.'app.version' = metadata.'app.version' + ".${buildNumber}"
        metadata.persist()
        event("StatusFinal", ["Application build number updated to ${buildNumber}"])
    }
}

This script will look for a file called ‘build-number.txt’, which is produced by the Build Number Stamper Plugin, and put in the working directory. This file is read by the script, and it updates the version number in application.properties, which will result in a war file produced by the build in the following format ‘-..war’, for example ‘todolist.0.1.34.war’. This way you can easily relate each build which bugs you might find, and avoids confusion, for example between testers and developers.

As an extra, we’ve show the current version of the application in the source of our HTML. We do this by reading the version number from the application.properties with the following tag:

This will display the version number of the application, and will allow you to relate issues to a build number, to avoid miscommunication about the deployed application!

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

Making Grails work behind an NTLM firewall (or using Grails without Internet)

Currently, I’m in the process of taking over an existing Grails project and migrating it to a different location. This location has been setup by me, so I have full control over it, except for one tiny detail: the Proxy Server.

The application used to work great, and installing plugins was a breeze: just do a good old ‘grails install-plugin fitnesse’, and the plugin would be installed. However, since this Proxy Server is not just any Proxy Server, but a Microsoft NTLM Proxy Server, we tried quite hard to make Java work with it, but at the moment the score is 1-0 for the Proxy Server. This gives us the following problems:

1) Our dependencies cannot be downloaded
2) Our plugins cannot be be downloaded
3) Some plugins cannot be installed anymore

Downloading dependencies
In our BuildConfig.groovy, we specified some dependencies like joda time and xstream. These dependencies are normally downloaded, but due to our Proxy issue, that doesn’t work.

Our solution to this is to use Nexus. Nexus is able to pass the firewall, and can download our dependencies. It serves as a transparent Maven proxy, which can also be used by Ivy, and solves our problem nicely. Besides, it’s a good idea to install a local Maven proxy anyway, mostly because of reliability, but also because of speed and making sure everyone uses the same dependencies.

Downloading plugins
Unfortunately, the Grails plugins are not in a Maven repository. However, a solution to this is to download the plugin zip (eg grails-fitnesse-0.2.zip), rename it (to fitnesse-0.2.zip), put it in the lib directory (even though I’m really not a fan of putting binaries in version control systems), and register it in the application.properties and BuildConfig.groovy.

The BuildConfig.groovy should look like this:

grails.project.dependency.resolution = {
    inherits "global" // inherit Grails' default dependencies
    log "warn" // log level of Ivy resolver, either 'error', 'warn', 'info', 'debug' or 'verbose'
    repositories {
        mavenRepo "http://172.30.60.106:8081/nexus/content/repositories/central" // nexus repo
    }
    dependencies {
        compile ('com.thoughtworks.xstream:xstream:1.3.1') {
            excludes "stax","stax-api"
        }
    }

    plugins {
        runtime ':hibernate:1.3.2'
        runtime ':tomcat:1.3.2'
        runtime ':fitnesse:0.2'
    }

As you can see, everything is configured here: the dependencies are here, the location to nexus is configured, and there’s a plugin closure. This “plugins” closure defines the plugins, and allows you to place them in the ‘lib’ directory. Be sure to rename the plugin zip to remove the ‘grails’ part, else it doesn’t work. Maybe these zips are also available from a Maven repository, which would solve the issue of putting binaries in our source control system, but I haven’t been able to find them yet.

Plugin installation
Some plugins, like the jQuery plugin, but also like my own plugin, the syntax-highlighter, download resources from the Internet when installing them. I haven’t been able to work around that yet, but my current solution is to just not use those plugins. I downloaded the jQuery javascript libraries and attached them to the project myself, and since we only use jQuery which does that, this solution is also an adequate fix.

Conclusion
So, as you can see, with some effort, it’s possible to make Grails work behind a Proxy Server, or without internet at all. This is a good solution for build servers, so you can have reproducible builds without the need for an Internet connection.

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

Most of us value the quality of our code highly. I do, and I expect you, as a reader, also to care for the quality of your code. Because of that, we write tests. Test which usually contain of 3 parts: the setup, the execution and the verification. And sometimes some cleanup. In this blog, I’d like to focus on the setup part, and try to get rid of some boilerplate code which can easily clutter our tests!

As I mentioned before, we usually write some setup code which serves as input for our test. This code can look like this:

def person = new Person(firstName:'Erik', lastName:'Pragt', age:30, married:true, hobby:"programming")

Writing this code can be time consuming, plus the code doesn’t reveal the intention of the test. What are the significant bits in this setup code? Is it the age? Is it my name? Or a combination?

To solve the creation of testdata, there are multiple patterns. On of those patterns is the Test Data Builder pattern. The idea behind the Test Data Builder pattern is that test data should be easily to construct, which is usually accomplished by chaining method calls (also called a Fluent Interface, or a Builder).

I’ve taken the concept to Groovy and made an TestDataBuilder Category. A Groovy Category adds functionality to your code within a limited scope (defined by the ‘use’ keyword). This enables the object which is used in the setup of the test to ‘borrow’ the method of the TestDataBuilder Groovy class, which allows it to construct itself with testdata. This is best demonstrated by the use of an example:

use(TestDataBuilder) {
    def person = Person.build()
}

By placing the creation of the data in the scope of the Category, the .build() method becomes available in for the Person class. Calling the build method will instantiate the Person class and populate all the fields with values which all equal true. So numbers will become 1, booleans true, Strings will get the name of the variable as a value, etc.

To use this in tests, and make the code more intention revealing, we can choose to populate some fields with our own values, and have the rest be generated.

use(TestDataBuilder) {
    def person = Person.build(name:'Erik')

    assert person.name == "Erik"
    assert age == 1
    assert married == true
}

The benefits of this is that you save a lot of setup code, and make the code a lot more clear and maintainable. If you want to use this, the code can be found on GitHub.com. (This class was previously called the ObjectMother, an error on my side. The change in name isn’t reflected yet in GitHub). Yes, there are some limitations, like it doesn’t handly all standard datatypes yet (like dates), nor does it populate associations (list with other objects) yet. Have fun, and any feedback on the code would be highly valued!

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

When I created the jworks.nl website in Grails, one of my requirements was to have a blog. Since the website is created in Grails, and the Grails application doesn’t integrate easily with a standard blogging solution like WordPress, I decided to create my own blogging solution. But, since this proved to be much more painful than initially anticipated, I migrated the website to a new design and decided to use WordPress instead. However, the plugin creation was fun to do, so I’m sharing my experiences here.

The current blog consists of little more than just a BlogPost class, and a Comment class. This worked pretty well, and I got the blog up in no time. However, my blog was found by the spam bots, and it got spammed a little. In the first couple of days, I got 3 spam messages a day, which could be easily managed by hand (however still quite annoying). Then, the spam load increased to 10, 100 and then a 1000 per day. Then I got really annoyed, and decided it was time for a solution!

My solution was not to choose a Captcha. Captcha’s tend to bother the user, making it harder to post feedback. That’s not something I’d look forward to. So I created my own solution, based on how WordPress does it.

WordPress has their own spam blocking service, which is called Akismet. The nice thing about Akismet is that their service isn’t exclusive for WordPress, but they provide an API so others can also use it! Which is exactly what I did: I created a Grails plugin for Akismet, called (surprise surpise!): Grails Akismet! I’m sure you didn’t see that coming!

It’s very simple to use. Just do a ‘grails install akismet’, add your site plus the Aksimet key to your Grails configuration, and you’re good to go! The plugin provides a Grails Service, which can be injected into most Grails artifacts (like services, domains, and controllers). By using the ‘check for spam’ method in the AkismetService, a call to the online Akismet service is done, which determines if the comment is spam or not. Very easy, and very non-intrusive!

So, if you want to get started with the plugin, please check out the documentation and let me know what you think!

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In one of the previous blogposts, I described a way to easily upload images by using Grails commands. This blogpost further builds on that, by providing a generic way to index uploaded documents. These documents can later be retrieved by using the search option provided by the Grails Search plugin.

To index documents, we’re going to need two components: one of these components is the Grails Search plugin, as described above. The other component is Tika. Tika is a document indexer library which can index many kinds of documents like Text, Word, Excel, Powerpoint, but also more exotic document formats, like MP3 and FLV. Combined with Lucene, this provides a powerful combo.

So, how do we get this to work? Well, first we need the application from the previous blogpost. I refactored it a little to represent the new generic format better. I renamed all references from image to document (so now we have a CreateDocumentCommand instead of a CreateImageCommand, as well as all other references), and I moved the upload.gsp to the views directory, since redirection and forwarding works better that way. After the refactoring was done, I first installed the searchable plugin by typing:

grails install searchable

I also needed Tika, so I added it to the BuildConfig.groovy in the grails-app/conf directory, like this:

        compile('org.apache.tika:tika-core:0.7')
        compile('org.apache.tika:tika-parsers:0.7') { excludes "xercesImpl", "xmlParserAPIs", "xml-apis", "log4j" }

Grails and extra XML libraries really don’t mix well, so to prevent some nasty class loading issues, better exclude them here. What we need to do now is to tell searchable to index our documents when we upload them. This is quite trivial really, we can just annotate the Document class (which in the previous application was called ‘Image’) with a Tika converter. Then we have to create the converter, register it in Spring, and we should be done. So, first things first: let’s annotate the Document class.

In the Document class, add a searchable closure with a converter for the byte array, like this:

package com.example.domain

class Document {
    byte[] bytes

    static searchable = {
        bytes converter: 'tikaConverter'
    }
}

We now need to create the TikaConverter. We’ll create a TikaConverter.groovy in the src/groovy/com/example/converter directory. The class should look like this:

package com.example.converter

import org.apache.tika.Tika;
import org.compass.core.converter.ConversionException;
import org.compass.core.converter.basic.AbstractBasicConverter
import org.compass.core.mapping.ResourcePropertyMapping
import org.compass.core.marshall.MarshallingContext

class TikaConverter extends AbstractBasicConverter {

    private Tika tika

    TikaConverter() {
        tika = new Tika()
        tika.maxStringLength = -1 // unlimited document length
    }

    @Override
    protected Object doFromString(String str, ResourcePropertyMapping resourcePropertyMapping, MarshallingContext context) throws ConversionException {
        return null // we don't need this
    }

    @Override
    protected String doToString(Object o, ResourcePropertyMapping resourcePropertyMapping, MarshallingContext context) {
        return tika.parseToString(new ByteArrayInputStream(o))
    }

}

This class is responsible for extracting text from any kind of document uploaded to the system. To make this converter available, we have to register it in the Spring context, and then we’re ready to go. To register the Converter in the Spring context, add the following lines to grails-app/conf/spring/resources.groovy:

import com.example.converter.TikaConverter

beans = {
    tikaConverter(TikaConverter)
}

Now everything should be set in place. Browse to: http://localhost:8080/document-indexing/upload/show to upload a document. Any kind of document will do, as long as it contains some text. When the uploading went alright, the document will be available in the Lucene index. You can test this by going to http://localhost:8080/document-indexing/searchable and type in some text from the document you have just uploaded. The result should be a result page similar to Google.

Last week, we got a nasty error in our application:

java.lang.ClassCastException: org.codehaus.groovy.runtime.GStringImpl cannot be cast to java.lang.String

The stacktrace was not helpful at all, and showed that the error was caused by a NullPointerException in Melody, which was a false message. After some searching around, we found the following code to be the culprit:

render(view: 'index', model: ["${command.class.simpleName}": command)

What happens here, is that a GString is used as a map key for rendering the model in the view. It’s never a good idea to use GStrings as a Map key, since the hashCode of GStrings are not guaranteed to be stable. A better approach is to use a String instead, which can be done using the following code:

render(view: 'index', model: [(command.class.simpleName): command)

The above is our current fix, and we changed our CodeNarc GString as Map key to a prio 1 rule. Whenever a prio 1 error occurs, our build will fail, preventing future errors like this.

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

Groovy 1.6 introduced some Groovy AST transformations like @Lazy, @Delegate and @Immutable, and there are more. This blog describes some pitfalls of using @Immutable, but to do that, a proper example of @Immutable seems well deserved.

A first introduction

@Immutable, like the name suggests, makes your classes immutable. This means that state changes from the outside, for example by the use of setters, are no longer possible. A good use case for this is when building Commands or DTO’s (also known as Command Query Separation). See for example the following code:

class CreateCustomerCommand {
  String name
  Date birthDate
  boolean married
}

The state of this class should not change. However, in Groovy, this is a bit harder to do than in Java, since Groovy uses a lot of reflection to invoke properties. So the following is quite valid in Groovy:

def command = new CreateCustomerCommand(name:'Erik Pragt', married:false)
command.married = true

Being able to do so could cause a lot of unexpected things, and a lot of unwanted behavior! To make sure this doesn’t happen, we can use the @Immutable annotation:

@Immutable
final class CreateCustomerCommand {
  String name
  Date birthDate
  boolean married
}

As you can see, two things have changed:

1. The class is now annotated with @Immutable
2. The class has been made final, which is a requirement for @Immutable

   When executing the above code again, an exception will be thrown:

   groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: married for class: CreateCustomerCommand

   Seems all pretty and cute, right? However, what would happen when someone with evil intentions (or maybe someone with an old Java habbit), decides to create a setter for this class?

   @Immutable
   final class CreateCustomerCommand {
     String name
     Date birthDate
     boolean married
   
     void setMarried(boolean married) {
        this.married = married
     }
   }
   

   Now, the code does execute without throwing an exception, and suddenly the code is mutable again. Unfortunately, there’s not much we can do about this, but it breaks the @Immutable contract. There is a small note on this on the @Immutable documentation page:

   “You don’t have to follow Groovy’s normal property conventions, e.g. you can create an explicit private field and then you can write explicit get and set methods. Such an approach, isn’t currently prohibited (to give you some wiggle room to get around these conventions) but any fields created in this way are deemed not to be part of the significant state of the object and aren’t factored into the equals or hashCode methods. Use at your own risk!”

   This text describes the other features of @Immutable: an automatic equals and hashCode method. Since the setter breaks the @Immutable behavior, this is excluded from the equals / hashCode generation. However, this only applies when the fields are private! So be aware of that!

   Conclusion

   My conclusion in this is quite simple: don’t expose state changes to the outside world whenever your intention is to be @Immutable!

   Happy coding!

   ]]> http://www.jworks.nl/2011/08/26/friday-repost-groovys-immutable-pitfalls/feed/ 0