Unique jobs at Land Information New Zealand

I am working at Land Information New Zealand (LINZ) for almost one year now and my finding is that New Zealand is so unique country, unlike any other. Over the time, I have slowly realized, that LINZ and especially the part where I am working - Location Information, is also very positively unique with it's culture and people.
There is a giant area of land and sea we are caring about, stretching from New Zealand over South-West Pacific to Antarctica. We are running unique free data publishing service. We are using and contributing to lot of Open Source software like QGIS, PostGIS, GDAL, Python and Linux. We have unique managers doing Debian packaging. We are singing Maori songs every Friday. We have numerous running clubs. We are facing unique nature challenges and Kiwis are still the most optimistic people around the globe.

Great news is, that if you want to know what I am talking about, there is a unique opportunity. We are hiring two very interesting positions - DevOps Database Developer and Spatial IT Solutions Developer.

Have a look and send us your CV.  

Automatic PostgreSQL tuning in Ansible

Once upon a time, there was a tool called pgtune, which I heavily used for setting reasonable defaults for automatically deployed PostgreSQL instances.

Unfortunately, this tool is not maintained for a long time, doesn't support new PostgreSQL versions and finally it was removed from Debian and all it's derivates.

I understand, that proper database configuration is result of deep knowledge of your data and queries, but in current world of automated deployment and containers, it is much more better to configure your PostgreSQL instance with some computed values based on current environment, then leaving default values designed for some very low memory system.

Therefore I have started to search Internet for some pgtune alternative and found great, new web based pgtune replacement developed by Alexey Vasiliev (le0pard).

Since I am using Ansible for my systems orchestration, I have decided to rewrite it's computing logic as a new Ansible postgresql_tune module.

Module requires following information to compute results:

• PostgreSQL version
• deployment type
• total memory
• maximal number or connections
• path to resulting PostgreSQL file
• path to resulting sysctl file

Example usage:

- name: Install PostgreSQL configuration
  postgresql_tune:
    db_version: "{{ POSTGRESQL-VERSION }}"
    db_type: dw
    total_memory: "{{ ansible_memtotal_mb }}MB"
    max_connections: "{{ POSTGRESQL-CONNECTIONS }}"
    postgresql_file: "{{ pgsql_config_dir }}/99-postgresql-tune.conf"
    sysctl_file: "/etc/sysctl.d/99-postgresql-tune.conf"
  notify:
    - service postgresql restart
    - sysctl postgresql reload
  become: yes

Source code is here and any feedback is very much appreciated.

New print from QGIS templates in GIS.lab Web

During implementation of print for our new generation of GIS.lab Web, our core developer Marcel came with very interesting proposal of interactive work with map content inside of print template produced by QGIS, which I haven't seen anywhere else.

The main idea is, that once print is activated in GIS.lab Web, it will download raw print output from QGIS Server using GetPrint request and will allow interactive visualization of map content directly in this template. You can zoom, pan and rotate map and see exactly how the result will look like. To get the better idea, see video below.

Print template in QGIS Print Composer

 Print template in GIS.lab Web

Video presentation of print in GIS.lab Web

Marcel and Rasto, thanks for very good work !

Upside Down

After more than decade of contributing my skills to my home country, my home region and town, I had a strong feeling, that I am missing the next mission. My company is already in hands of people, with more experience to deal with post-communist syndromes and I am free to start a new chapter.

Last year, I met great people from Land Information New Zealand, with passion to improve things, with lot of free software and open culture. Great challenge and huge space to move forward. No corruption and no dangerous immigrants :).

In few days, I am moving to Wellington. Packing 30 kg of life per family member and starting the great adventure. One thing for sure, I will greatly miss all the people left here. But I really believe, that at the end, it will proof as good decision for everybody. Including my new employer and Slovakia.
So Long, and Thanks for All the Fish.

GIS.lab Web - New Generation QGIS Web and Mobile Interface

Last year, GIS.lab team has decided to start a development of brand new version of our GIS web publishing interface known as GIS.lab Web. Our goal is to have a web application built on top of modern technologies, with very modern UI, optimized for mobile devices. Also, we want to build a hybrid Android application from the same code base, but with native UI interface. All of this standing on the shoulders of our favourite QGIS desktop and server software.

We have realized very soon, that if we would separate this project from the core GIS.lab system, we can produce generally usable QGIS web interface, usable with or without GIS.lab infrastructure. Maybe, even replace current version of QGIS Web Client.

This week, first alpha version of GIS.lab Web was born.

For a quick look, see the screenshots below,

GIS.lab Web

watch our short screencast,

GIS.lab Web

or try it yourself.

Thanks to all helping this happen. Looking forward for any feedback.

GIS.lab Web on GitHub - https://github.com/gislab-npo/gislab-web

FOSS4G Europe - hope we ALL meet together again

For me, it was very memorable week in Bremen. I came very excited and happy to home, to my family.

But, there is also one thing which makes me feel very bad.
At the same time, when all of us where enjoying this event, there was at least one person, speaking nearly the same language as me, who was probably little bit scared of how the return to home would look like. The home destroyed by madness and war.

Somehow I feel, that without a peace, nothing makes a sense. I hope, next year, we ALL meet together again and everybody will be returning to home with a smile on the face.

Tatiana, Ukraine, all peaceful people under the sun, god bless You !

High performance processing services in GIS.lab

This week I have attended GIS Ostrava conference in Czech republic with basic talk about GIS.lab. After my speech, I was asked to answer a question about missing Web Processing Service (WPS), namely PyWPS, in GIS.lab. Here is my short answer.

Currently, GIS.lab server acts as conventional server. It provides boot service, file, geo-database or chat server. From the first sight, it would be nice to implement also some WPS service there.
Since I have realized great and mostly free and unused power of bunch of modern multi-core client machines connected to GIS.lab network (which is increased by each connected machine), I have started thinking differently. I have started thinking about distributing load of all CPU intensive services also to all client machines in terms of load balancing. Modern multi-core machine wouldn't be hurt at all, if one of it's cores would be shortly occupied by computing task requested by some other member of GIS.lab network.

For better imagination, I would describe my idea on WMS service provided by QGIS Mapserver on which GIS.lab WebGIS application relies.
Currently, QGIS Mapserver is installed only on GIS.lab server and thus it can provide only limited number of concurrent requests. Moreover, too many concurrent WebGIS users could cause server overload and could slow down also all other server provided services.
Since all resources served by WebGIS are shared to all members of GIS.lab network and all members of network are running same operating system, it seems reasonable to implement load balancing service on top of QGIS Mapserver, which will redirect WMS requests randomly to all network members. In such way, we will improve situation drastically (disk load could be next bottleneck).
All we need, is to have all required server software installed on all machines in GIS.lab network. I am not in favor to install server software to all client machines. I am thinking about creating some processing software image (maybe LXC container) on server which will be mounted to all client machines and used in some chroot like environment.

This is simplified description of my idea, how to create high performance computing network from random office computers in a few minutes. Any comments ?

PS: Thanks for all listening my talk. Jachyme, once I will have this load balancing system, I will surely implement PyWPS on top of it.