Posts

The curious case of Grails’ generate-all

/in , /by

I’ve been using Grails for some years now, and my normal workflow is something like: grails create-app, followed by creating some domain class, followed by a a grails generate-all, and a grails run-app. It helps greatly in selling Grails, and people are usually quite impressed by the speed of having an up and running hello-world application in Grails.

However, since Grails 2.3.0, this is not the case anymore. The Grails scaffolding is now a separate plugin, and while that’s not bad per se, it is bad that it’s a compile time plugin. The compile time means that the plugin is only added to the project whenever a compile time target is used, and personally, I only get into the compile time whenever I do a run-app. And yes, you’re right, that’s after the generate-all, which, as you might have guessed, doesn’t work in Grails 2.3.0 before having run a compile target (and, not coincidentally, grails create-domain-class is such a target…)

So, should you run into this, running grails compile, even though it’s functionally not needed, solves your issues.

Upgrading Grails 2.2.4 to 2.3.0

/in , /by

I’m currently working on a trivial Grails application, running on Grails 2.2.4. However, two days ago, Grails 2.3.0 was released, providing many new features, like testing with Spock by default, Hibernate 4, etc. However, when upgrading to 2.3.0 using

I received the following error:

I turns out grails upgrade doesn’t fix this for you (Thanks Tim!) (and, starting from Grails 2.3.0, it’s deprecated and replaced by use-current-grails-version), and you need to fix this yourself. It turns out the fix is quite easy; just add replace your Tomcat and Hibernate dependencies by this:

And it will work. Hope it’s useful, and remember to carefully read the docs, I missed it.

Update (14-10-2013): If you are wondering why debugging doesn’t work: you’ll need to start Grails with run-app --debug-fork
IntelliJ Update (1-10-2013) : IntelliJ 12.1.5 has been released, which now has support for Grails 2.3.0.
If you are using IntelliJ (12.1.5 EAP at the moment), you’ll find out that it’s not compatible with Grails 2.3.0. There’s an issue for that, which has been fixed. Applying the patches from the Youtrack issue will fix the issue and will make Grails 2.3.0 working on IntelliJ 12.1.5.

Debugging Tomcat SEVERE: Error filterStart or listenerStart

/2 Comments/in /by

Today, we spent quite some time debugging our Grails application to find out why in ‘grails run-app’, the application worked fine, however, when running the application in Tomcat (6.0.35), an error popped up. The error not quite helpful indeed: SEVERE: Error filterStart.

To debug this, after some Googling, which pointed us into the wrong direction, I downloaded the Tomcat source and found the issue. All you have to do is change the logging.properties file in your TOMCAT_HOME/conf directory, and add the following:

This showed us the stacktrace, and pointed us in the direction of Spring Security Core (org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named ‘springSecurityFilterChain’ is defined), which will hopefully be a blog for tomorrow 😉

Friday Repost: Grails and Bamboo

/2 Comments/in , /by

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In our development environment, we use the complete Atlassian suite. We (or actually I) choose this environment because the products integrates so well together, but also because it integrates good with other products, like SpringSource Tool Suite, or, in our case, IntelliJ IDEA. And finally, the Atlassian suite has excellent support for building Grails projects, which, at the moment, is all we do. Simply put, we choose Atlassian because it rocks!

To integrate even better with the Atlassian Build Server (Bamboo), I’ve created a small script which will update the version number of the application with the Build Number. This way, we always know which build is running where. To do so, I created the following script.

This script will look for a file called ‘build-number.txt’, which is produced by the Build Number Stamper Plugin, and put in the working directory. This file is read by the script, and it updates the version number in application.properties, which will result in a war file produced by the build in the following format ‘–..war’, for example ‘todolist.0.1.34.war’. This way you can easily relate each build which bugs you might find, and avoids confusion, for example between testers and developers.

As an extra, we’ve show the current version of the application in the source of our HTML. We do this by reading the version number from the application.properties with the following tag:

This will display the version number of the application, and will allow you to relate issues to a build number, to avoid miscommunication about the deployed application!

Friday Repost: Making Grails work behind an proxy server

/8 Comments/in /by

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog.

Making Grails work behind an NTLM firewall (or using Grails without Internet)

Currently, I’m in the process of taking over an existing Grails project and migrating it to a different location. This location has been setup by me, so I have full control over it, except for one tiny detail: the Proxy Server.

The application used to work great, and installing plugins was a breeze: just do a good old ‘grails install-plugin fitnesse’, and the plugin would be installed. However, since this Proxy Server is not just any Proxy Server, but a Microsoft NTLM Proxy Server, we tried quite hard to make Java work with it, but at the moment the score is 1-0 for the Proxy Server. This gives us the following problems:

1) Our dependencies cannot be downloaded
2) Our plugins cannot be be downloaded
3) Some plugins cannot be installed anymore

Downloading dependencies
In our BuildConfig.groovy, we specified some dependencies like joda time and xstream. These dependencies are normally downloaded, but due to our Proxy issue, that doesn’t work.

Our solution to this is to use Nexus. Nexus is able to pass the firewall, and can download our dependencies. It serves as a transparent Maven proxy, which can also be used by Ivy, and solves our problem nicely. Besides, it’s a good idea to install a local Maven proxy anyway, mostly because of reliability, but also because of speed and making sure everyone uses the same dependencies.

Downloading plugins
Unfortunately, the Grails plugins are not in a Maven repository. However, a solution to this is to download the plugin zip (eg grails-fitnesse-0.2.zip), rename it (to fitnesse-0.2.zip), put it in the lib directory (even though I’m really not a fan of putting binaries in version control systems), and register it in the application.properties and BuildConfig.groovy.

The BuildConfig.groovy should look like this:

As you can see, everything is configured here: the dependencies are here, the location to nexus is configured, and there’s a plugin closure. This “plugins” closure defines the plugins, and allows you to place them in the ‘lib’ directory. Be sure to rename the plugin zip to remove the ‘grails’ part, else it doesn’t work. Maybe these zips are also available from a Maven repository, which would solve the issue of putting binaries in our source control system, but I haven’t been able to find them yet.

Plugin installation
Some plugins, like the jQuery plugin, but also like my own plugin, the syntax-highlighter, download resources from the Internet when installing them. I haven’t been able to work around that yet, but my current solution is to just not use those plugins. I downloaded the jQuery javascript libraries and attached them to the project myself, and since we only use jQuery which does that, this solution is also an adequate fix.

Conclusion
So, as you can see, with some effort, it’s possible to make Grails work behind a Proxy Server, or without internet at all. This is a good solution for build servers, so you can have reproducible builds without the need for an Internet connection.

Friday Repost: Preventing spam with Groovy and Grails

/1 Comment/in , /by

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

When I created the jworks.nl website in Grails, one of my requirements was to have a blog. Since the website is created in Grails, and the Grails application doesn’t integrate easily with a standard blogging solution like WordPress, I decided to create my own blogging solution. But, since this proved to be much more painful than initially anticipated, I migrated the website to a new design and decided to use WordPress instead. However, the plugin creation was fun to do, so I’m sharing my experiences here.

The current blog consists of little more than just a BlogPost class, and a Comment class. This worked pretty well, and I got the blog up in no time. However, my blog was found by the spam bots, and it got spammed a little. In the first couple of days, I got 3 spam messages a day, which could be easily managed by hand (however still quite annoying). Then, the spam load increased to 10, 100 and then a 1000 per day. Then I got really annoyed, and decided it was time for a solution!

My solution was not to choose a Captcha. Captcha’s tend to bother the user, making it harder to post feedback. That’s not something I’d look forward to. So I created my own solution, based on how WordPress does it.

WordPress has their own spam blocking service, which is called Akismet. The nice thing about Akismet is that their service isn’t exclusive for WordPress, but they provide an API so others can also use it! Which is exactly what I did: I created a Grails plugin for Akismet, called (surprise surpise!): Grails Akismet! I’m sure you didn’t see that coming!

It’s very simple to use. Just do a ‘grails install akismet’, add your site plus the Aksimet key to your Grails configuration, and you’re good to go! The plugin provides a Grails Service, which can be injected into most Grails artifacts (like services, domains, and controllers). By using the ‘check for spam’ method in the AkismetService, a call to the online Akismet service is done, which determines if the comment is spam or not. Very easy, and very non-intrusive!

So, if you want to get started with the plugin, please check out the documentation and let me know what you think!

Friday Repost: Indexing documents with Tika and Grails

/in , /by

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

In one of the previous blogposts, I described a way to easily upload images by using Grails commands. This blogpost further builds on that, by providing a generic way to index uploaded documents. These documents can later be retrieved by using the search option provided by the Grails Search plugin.

To index documents, we’re going to need two components: one of these components is the Grails Search plugin, as described above. The other component is Tika. Tika is a document indexer library which can index many kinds of documents like Text, Word, Excel, Powerpoint, but also more exotic document formats, like MP3 and FLV. Combined with Lucene, this provides a powerful combo.

So, how do we get this to work? Well, first we need the application from the previous blogpost. I refactored it a little to represent the new generic format better. I renamed all references from image to document (so now we have a CreateDocumentCommand instead of a CreateImageCommand, as well as all other references), and I moved the upload.gsp to the views directory, since redirection and forwarding works better that way. After the refactoring was done, I first installed the searchable plugin by typing:

grails install searchable

I also needed Tika, so I added it to the BuildConfig.groovy in the grails-app/conf directory, like this:

Grails and extra XML libraries really don’t mix well, so to prevent some nasty class loading issues, better exclude them here. What we need to do now is to tell searchable to index our documents when we upload them. This is quite trivial really, we can just annotate the Document class (which in the previous application was called ‘Image’) with a Tika converter. Then we have to create the converter, register it in Spring, and we should be done. So, first things first: let’s annotate the Document class.

In the Document class, add a searchable closure with a converter for the byte array, like this:

We now need to create the TikaConverter. We’ll create a TikaConverter.groovy in the src/groovy/com/example/converter directory. The class should look like this:

This class is responsible for extracting text from any kind of document uploaded to the system. To make this converter available, we have to register it in the Spring context, and then we’re ready to go. To register the Converter in the Spring context, add the following lines to grails-app/conf/spring/resources.groovy:

Now everything should be set in place. Browse to: http://localhost:8080/document-indexing/upload/show to upload a document. Any kind of document will do, as long as it contains some text. When the uploading went alright, the document will be available in the Lucene index. You can test this by going to http://localhost:8080/document-indexing/searchable and type in some text from the document you have just uploaded. The result should be a result page similar to Google.

Our Grails Stats: but what are yours?

/9 Comments/in /by

We’re currently working on a Grails project for over a year now, with, on average, around 8 people including testers. I thought it would be nice to share our project statistics, but I’m wondering how our statistics compare to yours.

Some information about our project:

  • Java Helpers: this is 100% generated (CXF/JAXB code).
  • Scripts: this consists of some scripts to set the build number, generate the above CXF code based on some WSDLs, and run our Javascript unit tests
  • Domain/Service: Most of our business logic is in the domain or service layer. There is still to much code in our controllers, and we are working on that to make our controller layer smaller.

I hacked the script to also display our GSP and Javascript files. The Javascript is not entirely accurate, since it also includes JQuery and some JQuery libraries. Our functional (Geb) tests are missing though. These are quite new in our application, but are taking a bigger and bigger portion of our tests.

So, I’m curious to see what your project statistics looks like. If you run:

grails stats

You’ll get the same format as above. Please post it as a comment, even if it’s only a small project, along with some project details you might find interesting to share, and let’s see if we can create some nice graphs out of this!!

Excluding fields and properties from marshalling using annotations in Grails

/1 Comment/in , /by

Recently the flexible and configurable marshalling system of Grails was mentioned on Twitter by Graeme Rocher as needing some more documentation. That was probably the outcome of the interesting blog post by J-Wicz on the matter. Rob Fletcher has also blogged about it quite some time ago.

The issue got my attention because in our project I needed to exclude some fields from marshalling on a DTO. The reason for that might be various, for me it was a fact that the DTO contained some intermediate data which was then used to determine a value that is being send to the frontend in a JSON format. One of the fields had to also be formatted so I created a getter for it but the original value was still marshalled which was not desired – you’re not using it then don’t send it to frontend, less data means better performance. For others it might be the fact that a marshalled object contains a big data structure which is not needed in the frontend.

I wanted the solution to be reusable so that it could be applied to multiple classes without the need to create a new marshaller for every one of them. I decided to mark the fields with a @DoNotMarshall annotation which would be then detected in my marshaller and the fields and properties annotated with it won’t be marshalled. I used the information from the abovementioned blog posts as the starting point.

So the first thing is the two annotations – one from marking the fields and accessors to be skipped from marshalling and the second to mark the class on which we want to configure the marshalling. I will explain why the second is needed later on.

The code for DoNotMarshall annotation:

And for the ConfigureMarshalling annotation:

Next we’ll need a test for our new marshaller:

Few things to note about the test class:

  • in setUp() we are registering our marshaller using JSON#registerObjectMarshaller(ObjectMarshaller<JSON>) method – you will also need to do the same in the bootstrap if you want to use the marshaller in a Grails application
  • AnnotatedClass and NonAnnotatedClass are our test classes for which we define which fields should be skipped when marshalling
  • there are two tests – one for testing if the fields and properties marked with @DoNotMarshall are not marshalled and one for testing if the @DoNotMarshall is ignored if the class itself wasn’t marked with @ConfigureMarshalling

And finally the marshaller implementation:

I decided to extend from ClosureOjectMarshaller (yes, it’s an ‘Oject’ marshaller, apparently a typo) which allows to specify a closure which is called with the marshalled object as the parameter and shall return another object (in my case a map) that will be marshalled. Thanks to that I don’t have to worry about any JSON building. I’m just returning a map from the closure where keys are property names and values are the respective property values and the result is later on JSONified for me.

The code inside the closure is a modified version of the code from GroovyBeanMarshaller with the sole difference in not marshalling the fields and properties on which the DoNotMarshall annotation is detected.

Finally note that I’ve overridden the supports method – thanks to that the marshaller will be used for any class that bears the ConfigureMarshalling and is a GroovyObject but for non annotated classes the original (in most cases the GroovyBeanMarshaller) marshaller will be used. As the AnnotationDrivenObjectMarshaller creates a map which is then marshalled it has the performance penalty of that extra step so if all the fields are to be marshalled it’s better to let the framework use the original marshaller.

To use this marshaller you need to drop it together with the annotations in the src/groovy directory of your Grails application and register it in the bootstrap as previously mentioned in the description of the test class. As the bootstrap is not available in test classes it might be a good idea to register marshallers in a static method of some utility class. You can then call it both from the bootstap as well as from the unit tests – this way you can easily replicate the marshalling configuration of a running application in the tests.

Happy annotating and marshalling!

Friday Repost: Remove Groovy MetaClass methods

/in , /by

The Friday Repost series are copies from my earlier writings. Since I don’t want to loose them, and they might prove useful to others, I’m reposting them on this blog

Whenever you are using metaclasses in Groovy or Grails unit tests, you might use Groovy’s metaClass power to mock the behavior of methods. However, Grails doesn’t fork the JVM for each test, so a nice side effect of this is that the mocked method will be available for the complete lifetime of the VM. Since Grails first executes unit tests (the test which usually contain the mocked code), and then followed by the integration tests, these tests can fail because you are maybe still using a mocked version of the method instead of the real one! This can often happen when mocking Grails methods, like dynamic finders, or named queries. The trick is to clean up these methods.

This is done by registering the metaClass when extending GrailsUnitTestCase. You’ll have to call the registerMetaClass(clazz) method, which does the following internally:

You can use this code in your tests in the following way, for example in the UserTests.groovy:

This way, the User class is registered with Grails to be cleaned up afterwards. This is done automatically in the teardown of the test, so you’re sure that no custom defined methods are available after running the test.

So, beware that whenever you use Groovy’s meta magic, be also sure to clean it up afterwards, since it might affect other parts of the code. You can check out my other blog post for a good solution to not forget the cleanup part!