Day 26

Today we made some promising progress on both experiments. Ron noticed that the ladder network performed exceptionally well on features generated by 4-5 layer SMCAEs and suggested that the ladder network may work best when the first layer reduces dimensionality instead of adding dimensionality. This theory was completely contrary to the results of the original paper and the "gold standard" MNIST ladder network, so I was a bit skeptical, but there was no reason not to try the same approach with 1-3 layer SMCAEs. So far, these kinds of models work marginally better than an SVM. I will withhold judgement until we do more trials.

On the SCAE/SMCAE side of the experiments, Ron suggested that since the stacks with more layers produce more features, the SVM used to classify may be prone to overfitting (explaining our poor results from last time). Hence, we should perform PCA before classifying with a fixed number of output components so that the number features remains constant. I haven't had the time/GPUs to fully test this, but it seems like it's doing a little bit better. Again, there isn't enough data to draw conclusions yet. Regardless, both theories are looking pretty good so far.

I also sat in on Titus and Aditi's test runs of their presentations for the MVRL crew. They got a lot of good feedback to improve their presentation skills, and I also learned a bit that I'll use for my presentation run tomorrow. We will see how that goes.

Comments

Post a Comment

Popular posts from this blog

Outline

Motivation/Background: HSI classification What is HSI? Comparison to RGB The advantages of HSI over other kinds of data Instant, remote, nondestructive data collection Hundreds of data points per pixel What is classification? Introduce using toy data set, like CUB-200 Applications of HSI classification Cancer detection Art authentication Food quality analysis Remote sensing Urban development agricultural monitoring environmental assessment Problems faced High dimensionality Scarcity of labeled data Method: MCAE and Ladder Network Feature Extractors Baseline: PCA Inspiration: Autoencoder Our proposed method: MCAE Classifiers Baseline: SVM Inspiration: Neural network Our proposed method: Ladder network Results : State-of-the-art Comparison Comparison of MCAE against PCA and state-of-the-art Comparison of Ladder Network against SVM and state-of-the-art
Read more

Day 2

Day 2 started with a very brief meeting where we went over the schedule for the next six weeks. Abstract, outline, presentation; two weeks per task. It was at this point that I realized the difficulty of making technical topics sound interesting, then I immediately dismissed the problem as one for my future self. Today was dedicated to Support Vector Machine (SVM) creation, evaluation, and optimization. I was unaware of many of the wonderful functions included in the provided libraries, and ended up doing the data reformatting less elegantly. It still worked, so I left it. When I finished up my SVM code, I tried to help Nate with the errors he was running into. The keyword here is "tried." Note that it's not easy to explain concepts that you figured out hours earlier and happen to be more complicated than they should be. Nate was a good sport about it though, and with Ron's help, he had a good idea of the solution by the end of the day. For the last part of the day,
Read more

Day 30

Last day before presentations. The SMCAE tests weren't going anywhere, so I fell back to the old data. I added it to my presentation, applied some finishing touches, and gave it to Joe. After lunch, Titus rehearsed his presentation in the auditorium, and Aditi, Anjana, and I gave him a lot of feedback. He managed to cut the time down significantly, but the organization of his presentation made it a little bit confusing, even to Aditi. I didn't feel like I had to rehearse mine, since I've been pretty good on time. At the lab, there wasn't much to do, since Ron was out for most of the day and all of the presentation work was done. Nate did receive instructions for the GRSS challenge from Ron, so I helped him with that for the rest of the day.
Read more