Now that 3.2 has been released with our great new users handbook, I think it's time to start working on developer documentation as well.
We've got plenty of outdated development pages in the old wiki (which I'm not going to link here, it's too embarassing :-)). Instead of bringing the old docs up to speed, I've decided not to describe the status quo, but instead to describe things which are to come.
I've been working on a new implementation of the workspace and resource concept in deegree, which fixes a couple of outstanding issues with the old one. In particular, it adds resource level dependencies.
In case you want to have a look at the code, check out the resource-dependencies branch on GitHub.
Since we decided to have developer documentation close by the code, I've added the description of the new concepts to the GitHub wiki. Have a look at the workspace documentation which describes the new concepts.
I'm unsure whether the code will make it into 3.3 (it'll probably be released in around four weeks or so), but 3.4 is definitely the target.
I invite all interested developers to have a look at the code and the docs. Let's make the workspace right this time!
Wednesday, April 10, 2013
Wednesday, March 20, 2013
Setting up eclipse working sets using maven
Setting up eclipse with maven using a combination of the eclipse plugin for maven and the deegree maven plugin works pretty good. Unfortunately, we've got more than a hundred modules in the deegree code right now, which means the list gets really long in eclipse.
Of course, eclipse supports working sets, which can be used to group projects. But it's a lot of click work to organize all those projects into working sets, especially if you remove and re-add the projects often (eg. if you switch branches often).
Since I really dislike clicking often, I've decided to add a new mojo to the deegree maven plugin which saves me the trouble of configuring working sets and adding projects to them.
Since deegree module names follow a certain pattern (deegree-<modulegroup>-<name>, eg. deegree-featurestore-sql) it's a good guess to use the module groups as working set names. For example all deegree-core-* modules will be put into the d3-core working set, all deegree-featurestore-* modules into d3-featurestore.
To set up the working sets using the plugin, run the following line in the deegree root:
mvn org.deegree:deegree-maven-plugin:1.19-SNAPSHOT:setup-eclipse-working-sets -Declipse.workspace=$HOME/workspaces/test/
Please note that this plugin is highly experimental, and has only been tested on the latest eclipse version. It might just destroy your eclipse workspace configuration! Use at your own risk!
Edit:
1.19 has been released, you can use this out of the box now. Please note that it is indeed experimental, sometimes the working sets seem to be lost for some reason. In that case, re-doing the procedure works (for me).
Of course, eclipse supports working sets, which can be used to group projects. But it's a lot of click work to organize all those projects into working sets, especially if you remove and re-add the projects often (eg. if you switch branches often).
Since I really dislike clicking often, I've decided to add a new mojo to the deegree maven plugin which saves me the trouble of configuring working sets and adding projects to them.
Since deegree module names follow a certain pattern (deegree-<modulegroup>-<name>, eg. deegree-featurestore-sql) it's a good guess to use the module groups as working set names. For example all deegree-core-* modules will be put into the d3-core working set, all deegree-featurestore-* modules into d3-featurestore.
To set up the working sets using the plugin, run the following line in the deegree root:
mvn org.deegree:deegree-maven-plugin:1.19-SNAPSHOT:setup-eclipse-working-sets -Declipse.workspace=$HOME/workspaces/test/
Use -Declipse.workspace to specify the workspace to modify, and optionally -Dworkingset-prefix to configure something other than d3- as a working set name prefix.
If you're generally interested in what the deegree maven plugin can do for you, check out the documentation.
Please note that this plugin is highly experimental, and has only been tested on the latest eclipse version. It might just destroy your eclipse workspace configuration! Use at your own risk!
Please also note that 1.19 of the deegree maven plugin is not yet released, you'll have to checkout my feature branch at GitHub and build it yourself (1.19 is probably going to be released on 2nd of April).
Edit:
1.19 has been released, you can use this out of the box now. Please note that it is indeed experimental, sometimes the working sets seem to be lost for some reason. In that case, re-doing the procedure works (for me).
Tuesday, January 29, 2013
deegree/Occam Labs in academia
Jürgen Weichand, author of the WFS 2.0 support for the Quantum GIS, recently published his master thesis (German only) on the topic of implementing and using INSPIRE DownloadServices.
While a large focus of the thesis lies on the client side (the WFS 2.0 plugin for Quantum GIS was developed in the context of the thesis), some server side solutions were also evaluated. Since UMN mapserver does not have a WFS 2.0 implementation, only GeoServer and deegree were considered OpenSource-wise, the GO Publisher WFS was chosen as closed-source example.
Looking at table 6.7 on page 67 it seems we're doing pretty well. I'm sure the upcoming 3.2 release will improve the situation even further. And it's one of the first publications explicitly mentioning us at Occam Labs as developers of deegree!
We probably could have made an even more prominent place in the thesis if we had a nicer web administration console ready (chapter 6.2.6 mentions this as the probable reason why GeoServer was chosen for an example implementation, simple features only!). At least with the deegree handbook nearing completion it will allow future thesis writers (and all others of course) to better understand how easy it is to set up a WFS serving rich features with deegree.
Edit: I did not mean to imply that GeoServer can't serve complex features (as Andrea pointed out it has been able to do that for years), it's just not as easy to do using the web interface (which Jürgen Weichand also pointed out in this thesis).
While a large focus of the thesis lies on the client side (the WFS 2.0 plugin for Quantum GIS was developed in the context of the thesis), some server side solutions were also evaluated. Since UMN mapserver does not have a WFS 2.0 implementation, only GeoServer and deegree were considered OpenSource-wise, the GO Publisher WFS was chosen as closed-source example.
Looking at table 6.7 on page 67 it seems we're doing pretty well. I'm sure the upcoming 3.2 release will improve the situation even further. And it's one of the first publications explicitly mentioning us at Occam Labs as developers of deegree!
We probably could have made an even more prominent place in the thesis if we had a nicer web administration console ready (chapter 6.2.6 mentions this as the probable reason why GeoServer was chosen for an example implementation, simple features only!). At least with the deegree handbook nearing completion it will allow future thesis writers (and all others of course) to better understand how easy it is to set up a WFS serving rich features with deegree.
Edit: I did not mean to imply that GeoServer can't serve complex features (as Andrea pointed out it has been able to do that for years), it's just not as easy to do using the web interface (which Jürgen Weichand also pointed out in this thesis).
Tuesday, January 15, 2013
deegree goes GitHub
There's breaking news in the deegree world, as deegree (at least deegree as in the deegree webservices) is now at GitHub! Have a look at the official repository yourself. And help yourself to a fork...
Our main reason for moving to GitHub (besides moving to git as a tool) was to make it easier for people to contribute. And that's exactly what happened! We hadn't even figured out how to properly work with pull requests and the famous git cheap branches and there was our first pull request from an external contributor!
I also extracted the deegree maven plugin into a separate repository. Have a look at the documentation I added in its wiki.
We decided to (gradually) move all developer related documentation to the wiki of the deegree webservices code base, so that should be the primary source of information for now. Links to the old wiki were added where appropriate.
So I hope to see more of those pull requests coming!
In a related note, on Friday (the 18th of January) I'll move some more deegree related projects like the old deegree 2 code base to GitHub as well, and then the svn repository will be shut down/made read only. In case some old project picks up again, I offer to migrate any of the old projects to GitHub, but for building the unchanged code base, the read-only svn should be good enough.
Friday, June 15, 2012
Bolsena 2012 #3: deegree rendering OpenStreetMap
deegree has been using OpenStreetMap as a background map for a couple of years now in its layer preview. That's already quite nice, but there's so much more that can be done with the OpenStreetMap data. To make a start I've begun to set up a workspace that makes use of OpenStreetMap data imported by using imposm.
To set up the data, I just followed the imposm tutorial. The workspace is configured using the production table schema, so make sure you run imposm --deploy-production-tables once you're done importing the data.
The workspace is located in the deegree svn (deegree-workspace-osm), so if you want to have a look, go ahead. It uses the GeoServer styles that were used in the FOSS4G benchmarking 2011. It still needs a little tweaking to improve the map quality (labels and stuff), but the styles worked out of the box. The labels along roads/waterways seem to be repeating without taking the gap configuration into account, so analysing the deegree renderer here seems to be in order.
Apart from that, this enables you to create your own maps based on the OpenStreetMap data. The new workspace is the place to start, you can just adapt the styles for your own needs or create your own layers any way you want.
We at Occam Labs plan to provide an online demo using the German and Dutch OSM extracts from Geofabrik. We already did some tests on these datasets, and the result was quite promising. So stay tuned for updates on this one!
Edit: Forget the comment about the deegree renderer not taking the gap into account. It didn't, but now it does, another successful Bolsena bugfix ;-)
To set up the data, I just followed the imposm tutorial. The workspace is configured using the production table schema, so make sure you run imposm --deploy-production-tables once you're done importing the data.
The workspace is located in the deegree svn (deegree-workspace-osm), so if you want to have a look, go ahead. It uses the GeoServer styles that were used in the FOSS4G benchmarking 2011. It still needs a little tweaking to improve the map quality (labels and stuff), but the styles worked out of the box. The labels along roads/waterways seem to be repeating without taking the gap configuration into account, so analysing the deegree renderer here seems to be in order.
Apart from that, this enables you to create your own maps based on the OpenStreetMap data. The new workspace is the place to start, you can just adapt the styles for your own needs or create your own layers any way you want.
We at Occam Labs plan to provide an online demo using the German and Dutch OSM extracts from Geofabrik. We already did some tests on these datasets, and the result was quite promising. So stay tuned for updates on this one!
Edit: Forget the comment about the deegree renderer not taking the gap into account. It didn't, but now it does, another successful Bolsena bugfix ;-)
Wednesday, June 13, 2012
Bolsena 2012 #2: Lenient SLD/SE evaluation in deegree
In preparation of setting up a new demo workspace (more on this later), I've been working on making the SLD/SE filter and expression evaluation more lenient. Until now, the parser already ignored invalid SLD/SE bindings within style files, but the evaluator of expressions (like PropertyNames) and filters was strict.
The new evaluation will try to fix missing namespace bindings for PropertyNames in expressions and filters. Such broken bindings can be hard to find, because even though the schemas require you to use qualified names/xpaths, normal schema validators will not find these errors.
One nice (and intended) side effect is that SLD files used to configure GeoServer can now be used without modification (unless some special GeoServer constructs are used).
Since it is still desirable to write valid documents, you can set a class to DEBUG in your log4j.configuration to see what PropertyNames are being repaired:
The new evaluation will try to fix missing namespace bindings for PropertyNames in expressions and filters. Such broken bindings can be hard to find, because even though the schemas require you to use qualified names/xpaths, normal schema validators will not find these errors.
One nice (and intended) side effect is that SLD files used to configure GeoServer can now be used without modification (unless some special GeoServer constructs are used).
Since it is still desirable to write valid documents, you can set a class to DEBUG in your log4j.configuration to see what PropertyNames are being repaired:
log4j.logger.org.deegree.filter.Filters = DEBUG
Monday, June 11, 2012
Bolsena 2012 #1: deegree on the couch 2
Following up on the 2011 Bolsena event, we got news from Volker Mische that we could try to adapt the GeoCouch feature store created then to the new 2.0 version of Couchbase.
This would allow for easy, transparent use of multiple Couchbase machines as a backend, and might also improve the speed situation of our last experiments.
Turns out it was only half a day of work to adapt the existing approach to the Couchbase one. With a little URL/URI fiddling the indexes could be used same as before, with only minor changes. We now used the approach to store the GML blob directly in the JSON document (base64 encoded).
Actually, using the Java libraries for Couchbase was pretty straightforward, and resulted in much cleaner (and much less) code than before for inserting. Since the Java libs use a custom protocol, inserting also seemed quite a bit faster.
We left the query interface to still use the old HTTP interface (with only minor URL/URI fiddling), and querying is around 50% slower than our PostGIS blob storage. We expect that using a few performance tweaks would result in approximately the same speed (we're not using any compression now, and using the Couchbase Java libs would probably also be an improvement).
So all said, the first day was already quite a success. Although the weather started out with rain and comparatively low temperature, the sun came out around the same time we did our performance tests. Must be an omen ;-)
In case you're wondering, Markus is currently working on the documentation. We're going to try to expose some test subjects here to the handbook... Actually, it seems that quite a lot of the people here have already used or are using deegree, so maybe we can actually find someone willing :-)
Stay tuned for more Bolsena news!
This would allow for easy, transparent use of multiple Couchbase machines as a backend, and might also improve the speed situation of our last experiments.
Turns out it was only half a day of work to adapt the existing approach to the Couchbase one. With a little URL/URI fiddling the indexes could be used same as before, with only minor changes. We now used the approach to store the GML blob directly in the JSON document (base64 encoded).
Actually, using the Java libraries for Couchbase was pretty straightforward, and resulted in much cleaner (and much less) code than before for inserting. Since the Java libs use a custom protocol, inserting also seemed quite a bit faster.
We left the query interface to still use the old HTTP interface (with only minor URL/URI fiddling), and querying is around 50% slower than our PostGIS blob storage. We expect that using a few performance tweaks would result in approximately the same speed (we're not using any compression now, and using the Couchbase Java libs would probably also be an improvement).
So all said, the first day was already quite a success. Although the weather started out with rain and comparatively low temperature, the sun came out around the same time we did our performance tests. Must be an omen ;-)
In case you're wondering, Markus is currently working on the documentation. We're going to try to expose some test subjects here to the handbook... Actually, it seems that quite a lot of the people here have already used or are using deegree, so maybe we can actually find someone willing :-)
Stay tuned for more Bolsena news!
Subscribe to:
Posts (Atom)