In a collection of previous posts, I introduced Imgup, a library I created and made available for distribution through PyPi. This has been my largest and most complex project to date, and it is still in development. It has gone through several iterations and versions, each building on and resolving issues from the last version, but also presenting new challenges. In what follows, I hope to provide a brief report on where the project stands, what issues it is facing currently, and how I aim to fix them.
Imgup: Refresher
It may be worth spending a moment or two to remind the reader what Project Imgup is all about. Imgup is a (relatively) simple library that has two primary functions. The first is to streamline the data reading and processing steps of an image classification project. This is the Uptools module, which is largely based around the Image_Dataset class. With Uptools, the user can quickly and efficiently read images from a local disk, rescale them, normalize them, split them into training, validation, and testing subsets, save them back to disk, and much more. The second function is to enable rapid and comprehensive in-place augmentation to the image data. This is the Augtools module. Below is an example of a random multi-transformational augmentation applied with Augtools to a sample image.
Recent Developments: Imgup Becomes Imgo
Since the last post, the source code for Imgup has been tested extensively on small data sets, with good results. Apart from a snafu relating to PyPi which meant that the name had to be changed from Imgup to Imgo, there have not been many significant changes. Small adjustments have been made to address minor issues, but for the most part the library functions as desired on these small datasets.
I am therefore pleased to announce that Imgo v.1.0.0 is now live and available to download through pip.
pip install imgoThe full source code, as well as demonstrations of the two modules (which at the time of writing may contain outdated references to Imgup) is available at https://github.com/elbydata/imgo.
Testing on Large Datasets
As the library is working well on small datasets (including the Boat/Car/Helicopter set included in the repo for demonstrative purposes), the time came to try it out in larger and more complex cases.
The original idea for Imgo came as a consequence of troubles with another of my ongoing projects: DeepShroom. DeepShroom is a 20-class image classifier built in TensorFlow whose goal is to recognize mushroom species. While this project has been in the works for rather a long time, it ultimately reached a performance plateau; achieving a testing accuracy of 81%. The goal from the beginning was to reach 90%, however this has not been possible yet.
Because of this plateau, I decided to start again and try a new approach. Part of this was a painstaking manual cropping and triage of the training images (of which there were approximately 10,000). This did improve the accuracy, because many of the images were found to be garbage, but only slightly. The validation accuracy jumped to about 85%, but the testing accuracy increased only to 82%.
I believe this plateau exists because of two reasons. First, the dataset is simply too small. It is unlikely that 10,000 images is enough to train a classifier, especially considering the fact that many of the mushroom species look remarkably similar (which is unnerving because some of them are extremely poisonous). Second, there was a significant class imbalance in the data. The largest class had more than 800 images, whereas the smallest class had less than 400 images. Below is the original class distribution for the project dataset:
Imgo: What’s Going Well
After trying in vain to obtain additional images, I started to wonder whether or not I could simply generate my own images. I had used image augmentation in the model training phase, so why not try and generate additional images through augmentation transformation and corruption functions? This was the birth of Augtools, which eventually lead to the following class distribution:
With Augtools, I was able to solve both the class imbalance and the small data problem by generating some new images, such as the following augmented Fly Agaric:
Augtools has shown itself to be quite useful in rebalancing and enlarging datasets. The new DeepShroom, training on this new dataset, has not yet been run, and so I am yet to see the effects. I am excited to see what will happen, and I will report on this as soon as I fix the remaining issues with Imgo.
Imgo: What’s Not Going So Well
While Augtools is performing well at scale, its sister module Uptools has been running into some problems.
It seems that in my efforts to make a comprehensive and image data management system, I have overlooked some of the key Python principles. The Image_Dataset class and its methods are sometimes quite complicated and convoluted, and seem not to be scaling well. The new-and-improved mushroom dataset contains 20,000 images, and since much of the Image_Dataset class initialization relies on for loops, the runtime for most of the Uptools functions is painfully long.
The dataset extracted from the mushroom images is a numpy array of the shape (20000,220,220,3), and so there are approximately 3 billion pixel values to compute at each step. Initialization of the Image_Dataset object takes approximately 5 minutes to complete, which is not terrible, since the test case rescales and normalizes all of the images, but is still rather a long time for what is not actually that large a dataset in the grand scheme of things.
Worse still are the tvt_split and save_as_np methods, which take approximately 11 minutes and 15 minutes to run, respectively. Again, this is not altogether a disaster because these functions will only need to be run once. It is, however, a use-limiting factor because performing the same tasks using Keras or SciKitLearn would not take nearly as much time.
I am therefore digging deep into the source code and trying to rectify any inefficiencies in order to speed up the runtime. Watch this space!
