Friday, January 31, 2014

High performance processing services in GIS.lab

This week I have attended GIS Ostrava conference in Czech republic with basic talk about GIS.lab. After my speech, I was asked to answer a question about missing Web Processing Service (WPS), namely PyWPS, in GIS.lab. Here is my short answer.

Currently, GIS.lab server acts as conventional server. It provides boot service, file, geo-database or chat server. From the first sight, it would be nice to implement also some WPS service there.
Since I have realized great and mostly free and unused power of bunch of modern multi-core client machines connected to GIS.lab network (which is increased by each connected machine), I have started thinking differently. I have started thinking about distributing load of all CPU intensive services also to all client machines in terms of load balancing. Modern multi-core machine wouldn't be hurt at all, if one of it's cores would be shortly occupied by computing task requested by some other member of GIS.lab network.

For better imagination, I would describe my idea on WMS service provided by QGIS Mapserver on which GIS.lab WebGIS application relies.
Currently, QGIS Mapserver is installed only on GIS.lab server and thus it can provide only limited number of concurrent requests. Moreover, too many concurrent WebGIS users could cause server overload and could slow down also all other server provided services.
Since all resources served by WebGIS are shared to all members of GIS.lab network and all members of network are running same operating system, it seems reasonable to implement load balancing service on top of QGIS Mapserver, which will redirect WMS requests randomly to all network members. In such way, we will improve situation drastically (disk load could be next bottleneck).
All we need, is to have all required server software installed on all machines in GIS.lab network. I am not in favor to install server software to all client machines. I am thinking about creating some processing software image (maybe LXC container) on server which will be mounted to all client machines and used in some chroot like environment.

This is simplified description of my idea, how to create high performance computing network from random office computers in a few minutes. Any comments ?

PS: Thanks for all listening my talk. Jachyme, once I will have this load balancing system, I will surely implement PyWPS on top of it.

Tuesday, January 21, 2014

Waiting for GIS.lab 0.3

GIS.lab 0.3 is going to be released in a few days. Here is a list of some interesting new features.

VirtualBox client support

Since 0.3, it will be possible to boot regular GIS.lab client session in VirtualBox and run it in full screen mode or in windowed mode along with applications of your favorite operating system (Linux, Windows, Mac).


Built-in IRC server

To allow effective communication between all GIS.lab users without a need of Internet connection, version 0.3 will provide built-in IRC server and prepared #gislab chat room.



Launching empty WebGIS project

Until now, WebGIS project was always based on some existing QGIS project. It means, that you had to create at least some basic skeleton of your project in QGIS, before it was possible to publish it on web.
Since 0.3, it will be possible to launch empty WebGIS project by visiting URL http://web.gis.lab in web browser or via applications menu GIS.lab > Empty WebGIS project. Empty WebGIS project will be automatically loaded with OSM and Google base maps and it will be prepared to start capturing initial data for your new project.


Data capturing and sharing in WebGIS

By my opinion, best new feature of 0.3 are improved data capturing tools, which allow to draw points, lines and polygons on top of WebGIS layers. All drawings are saved forever in GIS.lab database under unique ID and can be shared along with whole map via Permalink.

 

Simplified publishing of QGIS project in WebGIS

In 0.3 we have created QGIS plugin, which allows very simple publishing of created QGIS project to WebGIS. It provides possibility to choose desired base layers (OSM, Google) and automatically detects visible layers, zoom extent and used list of scales and more.


Detailed changelog can be seen here.

Thursday, January 2, 2014

QGIS development support in GIS.lab

One of a goals of GIS.lab project is to provide clean, uniform and predictable environment which can be easy used for GIS software development and testing. Support for GIS development is optional and can be enabled in configuration file before client image is installed.
GIS.lab project heavily relies on features provided by QGIS, therefore I have decided to put together some tools to automatize building and installation of this software from Git source code. Now we have two scripts which should be run on client machine.

  • gislab-dev-qgis-install - download latest QGIS source code and make install
  • gislab-dev-qgis - run QGIS installed from source code
These scripts are very simple. There is a lot of room for improvements. Any ideas ?

For more information see our wiki page.

Wednesday, January 1, 2014

GIS.lab runs on your Windows and Mac too

Since I have started my work on GIS.lab, I am receiving quite positive responses. In general, it seems that people like this idea, but in many cases they are not able to simply loose their favorite working environment by rebooting their machine to GIS.lab.  There is a clear demand to keep user's current Windows or Mac OS and run GIS.lab inside with all its benefits. In this post, I would like to explain possibilities of running both server and client sides on various operating systems.

GIS.lab server runs in virtual machine which successfully works without any difference on Linux, Windows or Mac. It is really up to your preferences which platform you will choose.

GIS.lab client can be launched as physical client machine where you will say goodbye to a system you are used to do your work until now, configure your machine to boot from LAN and enjoy a new GIS.lab environment. This option is, and always will be preferred, because it provides best performance.

In case you can't offer loosing your current system, there is already a possibility to launch GIS.lab client inside of your favorite Windows or Mac system in virtual client mode. This mode will boot the same client environment from LAN inside of VirtualBox machine and will keep your system. You can use GIS.lab client in a windowed mode, similar as it would be standard application or in fullscreen mode for best user experience.



For more informations how to install and use virtual client mode see our wiki page
Please, visit also our new Quick Start manual where you will find detailed step-by-step instructions how to install GIS.lab and create your first GIS project.


 

Thursday, November 28, 2013

GIS.lab 0.2 released

I have released GIS.lab version 0.2. This is a first major version which completes major ideas of this project. On this occasion I have also created simple web site.

Changes from version 0.1

  • renaming project to 'GIS.lab'
  • adding possibility to configure GIS.lab via configuration files (config.cfg, config-user.cfg)
  • adding possibility to restrict client machines by MAC address
  • refactoring of file sharing directory structure
  • adding possibility to mount external network storage on client machines
  • adding example data and QGIS project file
  • adding 'Conky' widget to client desktop, removing session manager from panel
  • adding support for custom projections configuration (added S-JTSK Krovak East/North).
  • adding WebGIS application for automatic QGIS project publication on web (formerly known as WMS Viewer)
  • adding additional software like Gimp, Inkscape, Google Earth ...
  • documentation update

Release notes

Sometimes after shutting down GIS.lab server ($ vagrant halt) it is not possible to launch it again ($ vagrant up). The reason is that GRUB boot loader is endlessly waiting for manual input.
To solve this problem shut down server machine using VirtualBox Manager GUI and uncomment line 'v.gui = true' in 'Vagrantfile' file. Start GIS.lab server again ($ vagrant up) and press ENTER in server's GUI console to boot.

Tuesday, November 19, 2013

New data capturing and collaboration tools in GIS.lab web application

GIS.lab WMS Viewer web application allows automatic publishing of user saved QGIS projects on web.
Few days ago we have expanded its possibilities with nice point capturing, GeoJSON export tool and possibility to share these data via permalink to other users.

Look how easy is to collaborate with other users and apply captured data back in QGIS.


Capture data and share map via URL
Export data to GeoJSON
Load data in QGIS

Sunday, November 10, 2013

How long it takes You ?

How long it takes You to deploy complete solution of GIS office including operating systems and software of central server providing geodatabase and web server, centrally managed 8 GIS editor's desktop computers and create three nice GIS projects on three different computers and publish them on web ?

Few days ago I have bought new laptop (Lenovo E330 with Inter i5 and 8 GB RAM).  I have installed it with Xubuntu 12.04 and encrypted LVM and made a test to accomplish this task with a help of GIS.lab (GIS.lab is an Open Source solution to create fully equipped, easy-to-use, pre-configured, centrally managed portable and unbreakable GIS LAN). 

The results where impressive:

  • installation of GIS.lab software dependencies from internet (Git, VirtualBox, Vagrant) - 4 minutes
  • downloading base Vagrant image - 5 minutes
  • GIS.lab installation ($ vagrant up ) - 20 minutes !
  • connecting my laptop to my company's LAN and booting GIS.lab on 8 computers from LAN - 7 minutes
  • creating three nice GIS projects in QGIS (data where prepared before) and publishing them on web using GIS.lab WMS Viewer - 12 minutes !
  • TOTAL: 48 minutes


Here are the results:



Do You know something which can faster ? Do You want to try it ? Any questions and comments are welcome.