Today, 10 years ago I received the e-mail approving the JFreeReport project on Sourceforge. This day also marks my first day as official project leader of the project.

10 years ago, JFreeReport was just starting out. A few weeks before that I added what would later be known as “Simple-XML” format to make it easier to define layouts. In the same year we got PDF export, Windows-Meta-File (WMF) rendering and report functions to calculate values and started gathering a community via the forums at www.object-refinery.com.

JFreeReport was written by David Gilbert. When I needed some printing capabilities for a customer of mine, I selected JFreeReport, as it’s code was clear, and it was easy to embed, had little dependencies and was free of weird runtime requirements (like the need for a JDK to ‘compile’ reports, whatever that means). JFreeReport was 100% in memory with the ability to run in any environment that had a Java runtime environment, regardless how restricted or resource-constrained.

And these days, we uphold that flag. JFreeReport, now Pentaho Reporting, still runs 100% in memory, with no need to swap report data to disk. The core engine is still small, even though the various data-sources can easily add up to 100MB or more, if you include them all.

Thanks to our vibrant community we had a successful decade. With remarkably minimal resources for development we now largely match what other multi-billion companies and their products can do. So thanks to everyone who contributed time, code and blood and tears to make JFreeReport … ahem .. Pentaho Reporting great.

And hopefully, ten years from now, we will be able to throw some flowers onto the graves of the dinosaurs of BI.

Oh yes! It’s that time of the year again. It’s community time – this year in Frascati, Italy.

As usual its late September, so time to gather again to see what exciting new developments are brewing in the community. From reading the talk announcements, it seems we will have another full scheduled packed with new projects, field reports from the trenches and the usual sharing of ideas, techniques and general chatter.

Surely, that is something that Twitter, IRC and Blogs or a Forum can never do. Random talks to random people that spark random thoughts that ignite random brain cells.

Back in June I wrote a posting on how to create your own report viewer for the Pentaho BI-Server. That article sparked quite some interest. Seems that many people feel the pain of working with a monolithic GWT application. My personal favourite bugger is the 4 minutes compile time each time I make a change. What is yours?

On Saturday I will give an 30 minutes talk about ways to retire the GWT viewer in favour of a better, more lightweight solution. Thanks to Jordan, the talk will contain amazing live code! It will contain XML. It may contain pixies and dragons. It will not contain XUL nor GWT.

Well, as a full-time code wrestler for Pentaho reporting and the Pentaho Report Designer I do not experience the adrenaline rush caused by real world projects. So I’ll be like Mr. Vampire – let me suck you dry for feedback, for things that did work well over the last year, and things that may not work well. And hey, hearing any feedback is good. It shows that all the code written does make a positive difference – not just for our sales team but for real world people with real world problems to solve.

In that way, I do look forward to all the talks. And – yes, I admit it – the Italian wine and Galliano and Grappa!

More often than I like to see it, I receive sample reports (usually as part of a bug report) that makes me feel sorry for the poor souls receiving the resulting documents.

Reporting is as much about numbers as it is about conveying a message effectively. Yes, it is possible to just send out a flood of numbers in a 100+ pages document and let the receiver mine for the right information. But in most cases, such reports simply never get read.

Remember: Data is not information.

Data is just random facts thrown out into the world. Information is data put into a context that makes sense to the receiver. Don’t send out data, send information.

A simple way of doing this is to begin from the end. Don’t think about what data you can push out, think about what information the receiver of your report needs and what they are supposed to do with it. And then either cut out everything that distracts them from the core message. Or, if you are required to put all your raw data into the document for legal or organizational reasons, at least produce a short summary with information and put the data into an appendix.

Choosing the right data for a maximal informative report is only one aspect of making good reports. The important parts need to be properly emphasized. Your reader needs to be guided by layout and structure. Your report must be understandable – hopefully at the first glimpse, or it will be filed under “read later”. And ‘later’ never happens anyway.

Good design is not a magical thing and you don’t have to be an expert to make a report that looks decent. Most design guidelines are rather complex and not necessarily practical. This video here is.

Idan Gazit’s talk “Design for developers: Making your Frontend Suck Less” contains some very basic (and extremely helpful) design advice. Even though he talks about web-applications, the majority of his advice can be easily applied to the design of reports.

Idan Gazit’s talk “Design for developers: Making your Frontend Suck Less”

I recently stumbled across an interesting article about the interaction between designers and programmers at the #AltDevBlogADay:

Designers are fundamentally interested in the properties of the end result – the visible ‘outer shell’ of the product. They don’t care about the technology inside it, beyond the observable effects (like it being fast enough, not crashing, etc). Their descriptions of the product, and how it should be, are thus all written in terms of these visible properties: which things the player experiences, when and where the player experiences them, and so on.

Programmers, by contrast, are focused on the internals of the product – they are tasked with filling in the gap between the outer shell provided by the designers, and the solid core of functionality provided by the platform. It’s like a translation process – translating high-level game concepts into low-level hardware operations – and so their explanations are in similar terms: which lower-level concepts are in use, how they are combined into large structures, and so on.

I work on the Pentaho Report Designer at day time, so I know the programmer’s perspective all to well. The report designer, I have to admit, is full of programmer level references and low-level nudges and screws. The Pentaho Report Designer is explicitly targeted as a power-user tool. Its primary purpose is to make all features and all attributes and styles accessible so that no one ever has to go down into a XML editor to create a particular report definition. And hey, we were successful in that respect. If there is a way to do something with the reporting engine, then today’s report designer enables you to do it.

In dark and moonless nights, however, my other, sinister side comes to life. I turn into a designer myself. Over the years I accumulated a few projects at the side where I needed to either create some artwork or plain web-designs for myself or those close to me. The mind set of me as a designer is indeed different – its the what instead of the how that matters most.

When I look at the Pentaho Report Designer with Programmer’s eyes, then well, we are almost there. Feature wise we move more and more to the advanced side of the universe. Once native cross-tabbing and multi-column support is included we actually match the much more expensive tools feature check-box by feature check-box.

But it is not about features.

Healthy discontent is the prelude to progress. – Mohandas Gandhi

Looking at the Report Designer with the Designer Head on, I see by far more areas that we need working on. Right now, designing a mildly complex report requires me to switch between data-views and design-views, subreport-contexts and various dialogs and long lists of meaningless attributes – and each time I have to go back to make changes, I have to think about “how” instead of “what”.

The good thing: Once we have recognized what is wrong, we can fix it.

Over the last few weeks, I brought in a couple of smaller changes to make my life easier. Here are a few of my favourites:

• A new colour chooser:

  Java’s built-in colour chooser is a great example of engineering. It does exactly what the spec says, but those who implemented apparently it never actually used it beyond simple “yes, works!” tests. For me, getting to the right colours was just plain painful to do. It really made me wonder why none of our users ever complained about it.

• Hiding and Un-Hiding Elements from the design view:

  Another example of engineering gone bad: to hide a element, you have to select that element, go into the attribute-view, locate the hide-on-canvas attribute, select the value, and finally select “false” in the selection-box that appears. Takes me about 10 seconds.Right-Click on the element and select “Show In Design View” from the context menu: 1 second.

• Selecting fields in elements and groups:

  With the new version you can see all the fields at once, and a double click selects these fields and brings them into the selection.

• Expert and deprecated options hidden:
  The list of styles and attributes on the various elements can be daunting. When I edit reports, most of the time I start searching for the properties again, even though I work with the tool for years now. So lets cut back the chaos – expert options are hidden now until explicitly asked for. And deprecated stuff is not for use anyway – so why show it?
• Convert any data-source into a table-data-source

  It was never easier to make your reports into completely static ones. Great for creating replication cases for bug-reports too 😉

• I added a screen-shot tool that captures either the current dialog or the complete desk and saves it as PNG in your home directory. That alone saved me a few hours while writing blog entries.

Personally, I have a hate-love relationship with the style and attribute tables. They are needed to expose all the settings, but they are neither easy to use nor a great help during the design process. One of the more older cases from before the initial Citrus/3.5 rewrite is PRD-871. It proposes to create a special property pane that hides the complexities of these generic tables and that brings the most commonly used properties together in a designer centric way. I am still itching to get this thing started.

For me 2011 will be a good year, if crosstabs hit the road running and if I can make Pentaho Reporting 4.0 a user-friendly designer that even mom and pop are able to use. To make that happen, I need to understand better, where we are doing good and where we should improve.

And for that I will need your help:

What works great for you when you use Pentaho Reporting? What are the features you cannot live without any more? What are your pain-points when you design reports? What features do not live up to your expectations? How can we improve?

Either tell me here, discuss it in the forum or come and give me a new case in our JIRA system.

Two more days and we will all have a blast of a time in Lisbon. This year with a large release close by I will solely concentrate on picking the brains of our fellow community members to see what bugs everyone most.

Developing a project as large and complex as Pentaho Reporting is like steering a large container ship. Many times we find ourself restricted by the existing API and the holy grail of API-level backward compatibility. Letting reports tell the datasources how to sort data? Forget it. Efficient crosstabbing within the corset of the layouting API – difficult. Multi-column layout without exploding memory consumption – you must be kidding!

The next release will be a major-number release called “4.0” – so I’m finally free to get rid of a couple of old mistakes and to comb through the API to bury a few dead bodies. For releases like this we can break up APIs and carve in more efficient ways to funnel information around. The sky is the limit once more.

Hopefully armed with loads of feedback and real-world needs and wishes we can then lay the architectural ground work for the next years to come. So don’t get curbed by reality – bring in your deepest desires. When I know your wants and wishes for in the next few years I can shape the API so that we can bring them all in.

Right now my personal list looks like that (In no particular order and not everything comes in the 4.0 release 😉 ):

• Truly Dynamic report parameter – enable/disable based on previous parameters
• Auto-Sorted data from datasources
• Native Crosstabbing
• CSS based styles system with all the perks it brings (selectors, external style definitions)
• Dynamically loaded subreports
• CDA integration (yes, Pedro, I know I’m slow)
• Add ProtoVis and Pentaho Charts to the designer (yes, I know, Pedro, I know …)
• Cell-based metadata support
• OLE-style inline-editor for subreports – see your content instead of a generic blue box
• Parameter layouting
• GeoBI components (Maps, visualizations)
• PDF files as report content
• JavaScript widget library for HTML and PDF
• Kill the WAQR
• A Lightweight Web-Based Report Designer (iWAQR)
• Better Inline-Subreport performance
• Multi-Column layout
• A smarter Report-Wizard

So how does your wishlist look like?

After years and years of studying various holy sources I am finally able to present my research results. Looking at the Qur’an (yes, Ministry of Homeland Security, I’m one of them apparently), the Popol Vuh, the Bhagavad-Gita and even a page or two of the Bible clearly proves that God (or Gods, as with omnipresent omniscient entities you can never be sure with whom you are talking right now) speaks Java (and not more than Java Language Specification 1.2).

Every reader of the aforementioned books first notices the convoluted and verbose use of syntactical elements. Every reader of Java source code first notices the convoluted and verbose use of syntactical elements. Of course, this is good, as it gives you time to think while reading the text, without overloading your mind with to many facts in to little time. Linguists found that redundancy in a language improves its error resistance and its understandability.

Along with that, there is a lack of shorthand constructs and cryptic sequences of unspeakable characters. Gods language is clean and simple so that every creature can hear it without getting knots in their brain. It is notable that some print media uses text known from lesser languages like *(x+=y++%8) to signal profanity.

The products and rules of God expressed in the holy books apply to the whole universe. Most of the texts make explicit statements about this “write once, run everywhere” principle. The creator of the universe made his rules .. well .. universal. Being the omnipotent entity he is he could have created a universe where he has no power. But to avoid imploding the brains of those who study the universe he obviously refrained from creating that paradox.

Holy (and maybe not so holy) men (and woman, but less in numbers and usually burning rather quickly if being overly verbose) pointed out that one can find the answer to any of life’s problems in those holy books, if you study the text long enough. Smart (and sometimes not so smart) men (and woman, but less in numbers but not burning these days due to worries about the CO2 footprint) pointed out that one can find the answer to any of the core programming problems in the fairly complete Java runtime library, if you study the JavaDoc long enough.

Sadly, even tho the Gods are Java and Java therefore is God, there is evidence that humans are probably created in C or C++.

The first thing you notice when you look at humans is their obvious need for manual resource management. Breathing is a classical example of allocating a resource (air) and freeing it after use (breathing out). There are numerous examples where failure of either allocation or release of the resource caused unrecoverable errors in the program execution.

Looking around the planet, it is also obvious that humans in general do not do a good job in freeing their resources properly. Garbage collection is virtually unknown even tho viable garbage collectors exist. The various resource allocation strategies and conflicts give cause to much grief and pain.

Looking at humans you can see a clear platform dependency. Although it is claimed that the same source or genetic code could be adapted to various environments, humans tend to die rather quickly (and sometimes in fairly interesting ways) if transplanted to a different runtime environment.

And last but not least: A freshly created human is void of any usable strategies and has to spend years and years to build up a reasonable runtime library (which then is over-adapted to the specific environment and fails easily when ported to a different platform (see the section on resource management above). Likewise the C languages themselves come with only the minimum standard library. Extra functionality has to be manually coded or brought in from outside. Humans call this process education, programmers “importing a library”. Different educations/libraries usually come with complex and contradictory requirements causing more vendor dependence and thus again pain and grief.

So humans: abandon your erroneous ways, repent and join the forces of god and stick to 100% pure Java.