First blog post of 2024. I want to welcome you all (Grady, Kevin, Yu, Het, Le, and Manal) to our summer.
This blog platform is a place where I hope that we can put formal updates on our projects. Having a forum where we write a little bit more formally about what is going on is super useful and has a different feel than Slack which is best for short form, informal questions.
For this summer, there are a few events that we may have regularly; at the moment that includes a reading group on Wednesdays at 1:30pm. I think as a group we should propose several days a week that we mostly try to be here (for example, M, W, Th). I expect to be here most days aside from some travel in early July and the middle of August.
There are some pages on this site that might be of interest, including this page of conference targets for our papers: https://blogs.gwu.edu/pless/upcomingconferences/
A quick note about the style of research group that I have --- i like to work towards unusual ideas and high impact papers. It's not my style to say "you must do X", or "I expect you to be here from 9am until 6pm". The students that have done the best with me in the past have taken that flexibility to not be stressed if they have to be out sometimes (to take care of kids, train for marathons, whatever), but then also work very hard when they can. I like that model. They have also tended to be the ones to manage projects, push new ideas to explore and push to finish papers. This is an important skill to learn, and there is value in learning it early.
In terms of this blog, I think one substantial post each week that captures what you've been working on, with nice figures etc... is really useful, and can help make paper writing be easier in the future. You can also write quick update blog posts each day that can serve as a diary, repository of cool datasets or papers you've found, etc.
I'm excited to have you all working with me; I hope that we all have a fun summer!
Category: Blogs
Glitter pieces which are measured off, predicted on, and the prediction follows what we would actually expect
There are quite a few pieces of glitter which are measured to have an intensity of 0 (or < 0.1 normalized) in the test image, but then predicted to have an intensity > 0.6 normalized. If we look at the intersection of the scan lines which correspond to the intensity plots of this centroid, we see that they intersect right around where the light was displayed on the monitor for the test image. So we would expect to see this centroid in our test image. 
Below, I have shown the centroid on the image (pink dot), and we can verify that we don't see a glitter piece lit there, nor are there any lit pieces nearby. If there was another lit piece close to this centroid, we may believe that we just matched the centroid found in the test image to the wrong centroid from our glitter characterization. This does not seem to be the case though, so it's a bit of a mystery.
UPDATE: we have ideas 🙂
- threshold intensities the same way I did in my gaussian optimization (finding receptive fields) in the camera calibration error function
- throw out gaussians that are "bad"..."bad" can mean low eigenvalue ratio or intensity plots not well-matched
Aperture vs. Exposure
I played around with the aperture of the camera, and took many pictures with varying aperture - as the f-number decreases (aperture becomes larger), we see that a lot more light is being let in. As this happens, it seems that the center of a glitter piece slowly shifts, probably because if the exposure remains the same for all aperture settings, some images will have saturated pixels. The image below shoes multiple apertures (f/1.8, f/2, f/2.2, ..., f/9), where the exposure was kept the same in all of the images. We can see that in some of the smaller f-number images, there seem to be some saturated pixels.
After realizing that the exposure needed to be different as the aperture varies, I re-took the pictures with f-number f/1.8, f/4 and f/8, and varied the shutter speed and it looks like the shift happens less if image taken with some exposure and low f-number look similar in brightness to an image taken with some exposure and a higher f-number. 
Next Steps
- fix the intensity plots in the camera calibration and re-run
- try throwing out weird-gaussian centroids and re-run calibration
- take iphone pictures with the XR, figure out which exposure works best with its aperture (f/1.8) so that we get images that look similar to our camera images (f/8, 1 second exposure)
Glitter Update – 10/24
Goal: I am working on accurately calibrating a camera using a single image of glitter.
Paper Title: SparkleCalibration: Glitter Imaging Model for Single-Image Calibration ... maybe
Venue: CVPR 2020
For the last week, I have been specifically working on getting the calibration pipeline and error function up and running using the receptive fields of the glitter. Now, in the error function, I am predicting the intensity of each piece of glitter, and comparing this predicted intensity to the actual intensity (measured from the image we capture from the camera). I am also using a single receptive field for all of the glitter pieces instead of different receptive fields for each glitter piece, because we found that there were enough 'bad' receptive fields to throw off the normalization of the predicted intensities in the optimization.
This plot shows the predicted vs. measured intensities of all of the glitter pieces we "see" in our image (many have very low intensities since most glitter pieces we "see" are not actually lit. Here we see that there is a slightly noticeable trend along the diagonal as we expect to see. The red points are the glitter pieces which are incorrectly predicted to be off, the green points are the glitter pieces which are correctly predicted to be on, the black points are the glitter pieces which are incorrectly predicted to be on, and the rest are all of the other glitter pieces.
I also tried using the old on/off method for the error function (distance from the light location as the error function) and found that the results were quite a bit worse than the receptive field method (yay!)
Goal: My next tasks are the following:
- search for the right gaussian to use for all the glitter pieces as the receptive field
- run the checkerboard calibration for my currently physical setup
- re-take all images in base and shifted orientations, re-measure the physical setup, take 'white-square' test images in this setup, and maybe some iphone pictures
Wheat Update — 10/24
(1) The one sentence scientific/engineering goal of your project
The current goal is to train a classifier to classify the "Awned/Awnless" classes.
(2) Your target end-goal as you know it (e.g. paper title, possible venue).
The further goal we imagine is to predict the "Heading Percentage", then train a model to find the "Wheat Spikes", the seed part of the wheat, to prove we can do it from UAV data.
(3) Your favorite picture you’ve created relative to your project since you last posted on the blog.
In the dataset, there is only one plot have the phenotype value "Mix", which means in this plot, the wheat have both awned and awnless phenotype. (It seems interesting, but we decided to remove them form the dataset first.)
(4) Your specific plan for the next week.
We imagine to finishing training the simple classifier this week, and see if we need to do more improvement work on it, or we can move on to the next step.
Visualization and Improvement of Using Manifold Ranking to Query Two Images at the Same Time
The car dataset I use contains 8131 images of 64 dimensions. These data have 98 classes which had been labeled form 0 to 97, there are about 60 to 80 images of each class. What I am trying to do is instead of using Euclidean Distance or Manifold Ranking to query only one image, use Manifold Ranking to query two different images but in the same class at same time, to improve the accuracy.
Example results of one node pair(Green border images means the image has the same class as the query nodes, red ones means not same class) :
Node pair [0, 1]:
Using Euclidean Distance query 2 nodes separately and their ensemble result:
Using Manifold Ranking query 2 nodes separately:
Manifold Ranking ensemble result and the result of using our Two-Node Query to query them at same time:
Node Pair [1003, 1004]:
Using Euclidean Distance query 2 nodes separately and their ensemble result:
Using Manifold Ranking query 2 nodes separately:
Manifold Ranking ensemble result and the result of using our Two-Node Query to query them at same time:
Node Pair [3500, 3501]:
Using Euclidean Distance query 2 nodes separately and their ensemble result:
Using Manifold Ranking query 2 nodes separately:
Manifold Ranking ensemble result and the result of using our Two-Node Query to query them at same time:
The improved algorithm is as follows:
1. Use faiss library and setting nlist=100, nprobe=10, to get the 50 nearest neighbors of the two query nodes. The query nodes are different but in the same class.(faiss library use Cluster Pruning Algorithm, to split the dataset into 100 nlist(cluster), each cluster has a leader, choose 10(nprobe) nearest clusters of the query point and find the K nearest neighbors.)
2. To simplify the graph, just use the MST as the graph for Manifold Ranking. In other words, now we have two adjacency matrices of the two query node.
3. Create a Link Strength Matrix, which at first is a 0 matrix has the same shape as the Adjacency Matrix, and if the there is a node near the 1st query point has the same class as the other node near the 2nd query point, then give a beta value to create an edge between these two nodes in the Link Strength Matrix.
4. Splicing Matrices, for example, the Adjacency Matrix of the 1st query point at top left, the Link Strength Matrix at top right and the transposed matrix of it at bottom left, the Adjacency Matrix of the 2nd query point at bottom right.
5. Normalizing the new big matrix as the Manifold Ranking does. However, when computing the Affinity Matrix, use the Standard Deviation of the non-zero values of the two pre-Adjacency Matrices only, to make the curve converge at ensemble result when the beta value is large.
6. Giving the two query nodes both 1 signal strength and others 0 when initialization. Then get the Manifold Ranking of all nodes.
The following plot shows the mAP Score of different beta values for all node pairs in the dataset, which means 8131 images in 97 classes, over 330k node pairs. I give the top ranked node score 1 if it has the same class as the query nodes and score 0 if not, the n-th top ranked node score 1/n if it has the same class as the query node and score 0 if not. As we can see, as the beta value increase, the mAP Score got the maximum value when the beta value at 0.7-0.8, which is better than only use one query node and the ensemble result of two query nodes.
My next step is to find if I can do some improvement to the time complexity of the algorithm, and try to improve the algorithm.
Using Manifold Ranking to Query Two Images at the Same Time
The car dataset I use contains 8131 images of 64 dimensions, which means in shape [8131, 64]. These data have 98 classes which had been labeled form 0 to 97, there are about 60 to 80 images of each class.
The algorithm is as follows:
1. Use faiss library and setting nlist=100, nprobe=10, to get the 50 nearest neighbors of all nodes. (faiss library use Cluster Pruning Algorithm, to split the dataset into 100 nlist(cluster), each cluster has a leader, choose 10(nprobe) nearest clusters of the query point and find the K nearest neighbors.)
2. Get all node pairs that in the same class without duplicate. For 2 nodes in a node pair, use the longest edge of MST to build a connected graph for Manifold Ranking, separately, as the Adjacency Matrix. Leave the two Adjacency Matrix keeping Euclidean Distance without doing normalization.
3. Create a Pipe Matrix, which at first is a 0 matrix has the same shape as the Adjacency Matrix, and if the there is a node near the 1st query point has the same class as the other node near the 2nd query point, then give a beta value to the edge of these two nodes in the Pipe Matrix.
4. Splicing Matrices, for example, the Adjacency Matrix of the 1st query point at top left, the Pipe Matrix at top right and bottom left, the Adjacency Matrix of the 2nd query point at bottom right.
5. Normalizing the new matrix and doing the Manifold Ranking to get the label of the highest scored node as prediction. Particularly, give the two query points an initial signal weight 1, other nodes 0.
The following plot shows the accuracy of different beta value for images in class 0. As we can see, as the beta value increase, the accuracy got the maximum value when the beta value at 0.8, which is better than only use one query point.
My next step is doing this process to all image classes to see the results, and make another plot that shows that either two close query points or far query points perform better.
Comparison of Manifold Ranking and Euclidean Distance in Real Data
This week, I use Manifold Ranking and Euclidean Distance to predict the label of certain nodes, and compared the results of these two methods.
The data is from Hong's work, it's a tensor in size 8131*64, which is an image data onto 64 dimensions. Also, I have the ground truth to every node, it is a dictionary that stores the labels for each node. The data structure is shown below.
...continue reading "Comparison of Manifold Ranking and Euclidean Distance in Real Data"
Optimization for Diffraction Grating Groove Data
Utilizing the scripts outlined in previous blog posts by Grady and I, we were able to create an optimization function to solve for the groove distance and orientation of the diffraction gratings on our holographic glitter sheet.
Using the data gathered from the plot, pictured above, we were able to get the angles needed for our diffraction grating formula
With the left half of the equation solved using the angles we got from our graph, and the wavelength solved using the method outlined in last week’s blog post about wavelengths, we were able to create an optimization function to solve for our missing information: the distance between the grooves and the orientation of the grooves.
However, there seems to be multiple combinations of orientations and groove distances that can produce the same angle combinations, therefore we need to use more information for our optimization function.
We decided to use all of the lines from a lit square on the monitor to one glitter board to see if acquiring more data to run our optimization on would provide more specific results.
However, there are still multiple combinations of distance and orientation for the grooves that result in near-zero values for the error value. To combat this, we are looking for more parameters for our optimization function that would add constraints to the answers we receive, such as a minimum or maximum value for the groove distance. We have begun working on a script that will look at all pixels and their angles to light squares on the monitor, rather than just one pixels’. Hopefully this large amount of data will produce more specific results from our optimization function.
Recurring Ranking on Data Manifold and Refine with MST
Last week I recurrent the paper: Ranking on Data Manifold.
I created these 2-moon shape data randomly and added some noise on it. The left plot has 50 random 2-moon shape nodes, while the right one has 100 (the following plots correspond to these two).
...continue reading "Recurring Ranking on Data Manifold and Refine with MST"
A TUMAP issue
Last week, I find a problem of UMAP which is that if the high graph of a embedding representation is not connected, like NPair result on CAR training dataset, the optimizer of UMAP will keep push each cluster far away, which doesn't matter in visualization, but in TUMAP, we need to measure the loss of each map.
So, we try some different way to avoid or solve this problem.
First, we compute KL distance of normal UMAP with TUMAP result instead of compare their loss.
Second, we try to optimize the repulsive gradient of edge in the high graph instead of each connection of every two points. But the result of this method gonna wired.
And, I try add a 0 vector into the high dimensional vectors, and make it equally very far from each points when constructing high-D graph. It doesn't work.