Seeking way to tile/pyramid 4 million images using gdal for geoserver WMS?

by PCL   Last Updated July 11, 2019 21:22 PM

My scenario consist of 4 million geolocated images each measuring 1536x1536 pixels in size.

What is the most efficient way to pyramid these images?

My end goal is to serve these images using geoserver via WMS requests.

I have tried creating a virtual dataset of all images and putting them into gdal_retile.py. I have also tried merging each tile with a transparent "null" image that covers the globe then pyramid the result. Both methods seem like they will take forever and make me think there is a better way.

I also have the requirement of adding additional images in the future.

Is there an easy way to do this without having to regenerate everything again?

I am new to geoserver and gdal.

Tags :

If you are in a windows environment you can create a batch file:

@echo off
for /f %%A in ('dir "c:\your_Path\*.tif" /b/s') do (
echo %%A
"c:\path\to\GDAL\bin\gdaladddo" -ro %%A 2 4 8 16 32
)


This is a batch file that says "for this folder and every subfolder, every file with the '.tif' extension run gdaladdo". You will need to change c:\path\to\GDAL\bin to match your GDAL install location, you will also need to change .tif to the actual extension of the files.

Change the c:\your_path\ to where your folder of 5k is and run the batch file. You can run one per CPU thread in a separate cmd window. There are other options that can be added but let's just keep it simple.

important if any of the paths contain spaces you will need to quote them: "c:\path with spaces\files\" or DOS will spit the dummy, if the file names themselves contain spaces you will need to do something in python.

Michael Stimson
May 21, 2014 05:54 AM

Related Questions

Updated March 13, 2017 08:22 AM

Updated September 24, 2019 11:22 AM

Updated December 27, 2018 12:22 PM

Updated June 09, 2017 14:22 PM

Updated April 07, 2018 11:22 AM