nxSoftware Blog

The next update of the Arctic Reports system shall be the last update before I spend some serious time upon the documentation and some additional tutorials. Following this, work will begin on the first of a planned series of major major updates. These will include (in no particular order) : a scripting engine for the creation of code modules for our reports, a charting module, bar-code support, native PDF support and so on. The first will of course be the scripting engine.

The update currently in development brings, amongst other things, variable height report sections and 'anchorable' controls (allowing controls to dynamically resize to match the report section in which they are housed etc. Currently, the only way of altering the height of an individual report section dynamically is through a user-defined function and these are not particularly easy to set-up or maintain.

This update is actually complete but for a facility by which report sections can automatically grow to accommodate their content etc. For example, some control or other may take it's content from a database field and the control (and the parent report section) can be made to resize to exactly match the text etc.

This is one of the most frequently requested features for Arctic Reports and I have to say that, whilst I was always a little dubious as to the worth of such a facility, I am nevertheless very impressed with the outcome and the results. Having report sections able to grow or shrink dynamically is actually very very useful. Quite why I didn't see this earlier is beyond me? I have applied variable height sections and controls to a tiled report producing address labels and the resulting report looked far far better than I imagined it would. An invoicing report was made to display a table of records with rows of varying heights. It all blended perfectly.

Yes, a nice facility... and one I should have added some time ago.

Better late than never though.

Stephen.


**EDIT : well, the 'growable' report sections and controls is complete and working very well. Makes a hell of a difference actually. The work threw up so many bugs in the nxReport core module which had hitherto passed unnoticed that the damn thing nearly collapsed under the strain! All fixed now though.

Next up I shall add an ability to take images directly from a data-source (as well as other sources) and then I shall release the new versions of nxReport and Pyrex. Shouldn't take too long. (**EDIT : done!)

Hi,

well, added support for PostgreSQL databases to the core nxReport module without a hitch. I must say that PostgreSQL is a fantastic DBMS, one that I shall make use of whenever I am in need of a very sophisticated but easy to use server.

Now, adding PostgreSQL suppport to the Pyrex designer... that is proving somewhat trickier. More accurately, it has thrown up a limitation of the database operations currently performed by Pyrex. For example, when the user selects a subreport to edit then Pyrex rushes off (unless you turn this setting off) to retrieve a list of data-fields obtained from the underlying data connection (if one is defined for the subreport in question). Now this is fine, in principle at least. However, when I feed Pyrex a report taking data from a PostgreSQL database and switch the server off, then there is a huge delay whenever we select a subreport for editing. A delay brought about by Pyrex attempting to make the connection to the offline server. Of course, a similar story will unfold if using, for example, an ODBC connection to an off-line server etc. And the fact that I overlooked this is really a symptom of my general inexperience with working with large DBMS'.

Still, a solution is always to hand if you think long and hard enough.

My first attempt at a solution was to throw the connection routines into a separate thread and display a 'please wait' type dialog if the initial connection took an inordinate amount of time. The dialog gives the user the chance to cancel the connection and get back to working with the underlying subreport etc. This worked fine... until I smashed head on into an ADO connection!
The problem with my first 'solution' was that the connection to the database was being made in one thread and then the subsequent 'connectionID' was being passed back to the main process for use in retrieving the aforementioned fieldnames etc. So, we have a situation in which a database/connection handle was being passed between threads.

This works fine for SQLite and PostgreSQL (not sure about ODBC). But ADO...... crash bang wallop! At least with some OLE-DB providers the result was ... crash...! Actually I was suprised it worked at all for some providers (worked okay with Jet) considering that we are actually passing COM objects between threads in an environment initialised with a single threaded apartment COM model (by Purebasic itself). This effectively prohibits the passing of COM objects between threads. A bit of Googling and I discover that ADO actually side-steps the threading apartment and uses a free-threading model. This means that whilst ADO in general allows connection objects to be passed between threads (in some limited fashion) we are then reliant upon the underlying OLE-DB provider being threadsafe, and as I have learned... not all providers are threadsafe!

A second solution was therefore required. A more robust one! And that is what I am working on right now, right at this minute in fact. A small library which shoves all database routines into separate threads, meaning that I can still throw up a dialog allowing the user to cancel an attempted connection whilst that attempt is still underway. And since all operations are undertaken within the same thread which instigated the connection, there will never be a need to pass any kind of object between threads, least of all a damn COM object!

Hi,

Pyrex 1.40 is finally complete and it is a huge upgrade to the report designer, bringing, amongst other things; copy and paste of controls, undo/redo, short-cut context menus, control moving/sizing via the mouse (as opposed to the cursor keys etc.) full listings of subreport datasource fields alongside the design canvas (which can be dragged to the canvas) and so on. And let me say that all of this makes one hell of a lot of difference to the report design process!

Pyrex is effectively a new designer as it is so much more complete (in terms of it's features) than the earlier versions. It is, in my opinion at least, beginning to give more established designers a run for their money now!

Next will be relatively minor modifications to both the core nxReport module and Pyrex to bring in variable-height report sections and control anchoring which will then complete what will by then be a quite powerful system indeed. Following that, work will begin on what will be a series of major upgrades to the Arctic Reporting system to bring in, amongst a host of other things, a scripting component and then bar-code support (without using the cursed bar-code fonts!)

So, for me at least, exciting times ahead since this project is an absolute joy to work on! Challenging, demanding,... and very useful at the same time!

In terms of Pyrex 1.40, yes it was a lot of work. About 2 months full-time. The hardest element was of course the undo/redo facility, although the truth is that I designed Pyrex from the very outset with this capability in mind, constructing the core of the designer around suitable Memento design patterns. A relatively simple application of my Demento utility then made for a quite painless progression until full undo/redo had been added. I am still testing this however, as there are a lot of subtleties to the entire process in ensuring that the undo/redo is completely synchronised with the Pyrex GUI. For example, if we undo a control resizing then the main property box must reflect these changes if the affected control's properties are being displayed at the time of the undo etc. Very fiddly!

Stephen.

Hi,

fantastic progress on the update to the Pyrex report designer (Pyrex 1.4) is about to be interrupted by a short camping holiday. Only a short break mind to ensure that I am back for the World Cup! A longer break will follow the World Cup!

For those interested in a quick peek at the forthcoming version, I have prepared a screenshot : Pyrex 1.4 screenshot

Note the list of dataset fields on display on the right-hand side of the main GUI. Pyrex 1.4 will attempt to locate the exact field names for the subreport currently being edited (assuming the subreport utilises some kind of data-source) and this will make designing reports a lot easier in many cases. I have answered many questions in the past regarding 'missing control/field' errors and in the majority of cases these have boiled down to the developer simply not identifying the correct field-name(s) as reported by the underlying data-source. These fieldnames are now displayed, where possible, by the Pyrex designer.

Moreover, you can simply drag these field names to the design canvas and appropriate bound controls will be immediately created. Saves having to drag out a control and set the source properties by hand!

Stephen.

Back working full-time on Arctic Reports now and I had forgotten how much fun it is!

First up is what will, in the grand scheme of things, be a minor update to Pyrex (Pyrex 1.4). My previous blog entry claimed that this will be a major update, but that status has been downgraded in the light of the work now planned for both the Pyrex designer and the nxReport core module.

Pyrex 1.4 will have the following :

• Double-click controls to bring up the expression builder. (Done)


• Reposition controls by click-and-drag etc. Presently, we use either the cursor keys or the arrow buttons. (Done)


• Dymanically resize controls by click-and-drag etc. Presently, we use either the cursor keys or the arrow buttons. (Done)


• List dataset fields alongside the design canvas and allow drag/drop of ready-made controls bound to the underlying dataset field etc. (A late addition to the new feature set for Pyrex 1.4 - partially complete)


• Set report parameters to allow parameterised reports to be previewed from within the designer. (A late addition to the new feature set for Pyrex 1.4 - partially complete)


• Full cut and paste of individual controls. (Partially complete)


• Undo/redo. (Partially complete)

Numerous bugs have already been fixed along with many minor modifications.

After the release of Pyrex 1.4 will come a major revamp of the nxReport core module. Included in this will be a new file format (one based upon OLE structured storage) which will be far more extensible than the current format. This will break existing reports, but there will be a very easy to use utility for converting between the old format and the new forthcoming one. I shall probably integrate the utility directly into the Pyrex designer.

At this point, I shall launch into the promised scripting engine and will not look up until that is complete. The scripting engine will come with many enhancements to the nxReport core module to allow scripts to leverage the full power of the core library. This will open up a lot of possibilities, some of which can only be found in very expensive reporting tools. This will, I believe (and hope) elevate Arctic Reports to an entirely new level. Of course, the Pyrex designer will, at this point, be upgraded to offer full code editing facilities so that developers can design their reports and write their scripts all within a single environment.

After this a full charting module (which will take full advantage of the planned changes to the underlying report file format). This is something I am in need of as part of a future project which I am committed to.

A busy time ahead.

Currently working on Pyrex 1.4 which will be a major update in that it will offer copy/cut/paste of controls aswell as undo/redo facilities (and not before time perhaps! )

I say will offer, but this does come with a somewhat tentative proviso where the undo/redo facility is concerned ... Users may have to wait until version 1.5 for this if it proves to be too problematic. Sure a lot of the code is already in place for ths undo/redo facility, but this still represents a major undertaking anyhow.

Whilst I am re-acquianting myself with the Pyrex code (32000 lines of Purebasic source without the nxReport core module), certain obstacles do present themselves to achieving a full undo facility. For example, if a user deletes an entire subreport together with all companion controls, how best do we undo that little operation? How do we reinstate the subreport (and all of it's properties and those of other subreport's which depend on this one) together with the possibly hundreds of companion controls together with the dozens of properties for each control together with their places within the object tree... A thorny problem indeed!

My initial thoughts are to make the deletion (and therefore the addition) of subreports an action which cannot be undone. I guess I have to draw the line somewhere. Deleting multiple controls will certainly be undoable, but entire subreports...... Perhaps I shall indeed exclude deleting subreports from the undo/redo facility and think about adding this at a later date should there be a demand for it? It sounds like a plan to me!

Copy and paste of controls (not subreports) presents a different challenge, one that would seem to be relatively straight forward. Again this must be undoable and so I need to code this in such a way that it fits in to the existing undo/redo framework (as it exists at the moment). It occurs to me that the information required to be stored on the clipboard is identical to that required to be stored in an undo stack when a control is deleted anyhow and so there is a nice link between the two areas of focus of this next update.

Internally, Pyrex operates like a scripting engine in that 'commands' are despatched to a central 'hub' for processing. For example, when the user moves a control then a suitable command is sent to the central hub informing it that the underlying control should be moved etc. An undo/redo facility simply needs to place these commands within a stack and despatch these commands when appropriate. Something similar is true for copy/paste. When a control is to be copied, a suitable bunch of these 'script commands' are to be tied together and written to the clipboard. A paste operation then simply retrieves this bunch of commands, modifies them as appropriate (e.g. the command for setting a control's ID will need to be altered to give the new control a unique ID) and then sends them to the central hub.

Perhaps not the fastest way of implementing copy/paste and undo/redo, but certainly amongst the most convenient!

Well, ExGRID is complete (and with a fully detailed user manual!)

This took somewhat longer than I had originally intended (especially the manual!) Though, in all fairness, ExGRID does come with more features than I had originally intended for it, quite a lot more in fact. And the OOP programming api does make the whole business of creating complex grids a lot simpler.

Of course, with the new fixed column of row header cells and all, ExGRID (like the original EsGRID control upon which ExGRID is based) is still based upon a Windows ListView control and so of course is subject to the same limitations where they exist. Still, I am quite proud of ExGRID and look forward to using this control myself in a future application or two.

The question does arise now, though, of whether I shall create a new custom grid control at some point? One based, not upon a Windows ListView control, but one built entirely from scratch? This has been asked a few times now and I have to say that I certainly will be creating a completely new grid control at some point in the future, one that will not have quite the power of EsGRID/ExGRID, but one that will occupy a nice little niche anyhow. A grid built from scratch will not have any limits (aside from physical memory etc.) in terms of the number of columns which can be added in that performance will not be hit in quite the same way as a typical ListView control.

So yes a new grid is on the 'to do' mountain, one that will not replace ExGRID, but one whose underlying simplicity should give it the edge in more data-intensive applications. And with all the lessons learned in creating a grid control based upon a Window's ListView, let me tell you all right now that a grid built from scratch will be a damn sight easier to code!

But this 'new grid' control is for another day.

Welcome to the nxSoftware blog.

Here I will post some occasional thoughts regarding my software development and the various projects with which I am involved. Perhaps a few progress reports as well along the way, when appropriate.