@zunair_charolia

I have highly refined the depth map algorithm, which has helped my project with the final result a ton. The computer keys are still the worst-case scenario from what I’ve tested, but it even handles them much better than the previous method. In order to achieve such a drastic difference, I have used a correction algorithm and an infill algorithm (see previous journal where parts were black (undefined). One of the big problems with the keyboard was that it would create a halo. Bokeh is still sharp, but it is also unfocused. This caused the SML algorithm to fail and create these “rings” around the true depth. To mitigate this, the correction algorithm calculates the edge of it’s own region. If a region is thick enough (this is a constant within the code), it is marked as trustworthy. Thin ribbons will fail this test.The black parts (missing data) is solved using an infill algorithm. It uses the original photo as a guide and assumes that if two adjacent pixels have the same color, they probably belong to the same object and should have the same depth. For every unsolved pixel, it looks at it’s neighbours and calculates the “color weight” (closeness in color). If the pixel has a high color weight, it likely belongs to the same object. If it has a low color weight, it is likely something like an edge. This process is repeated 40 times to ensure the entire frame is filled.

After using AI, I have completed the blurring logic. (For this in particular, I used AI a little more than usual. I’m not a fan of it myself, but I’m in a time crunch, and the logic is a little complicated.) However, the results have turned out… Interesting. I will definitely have to double-check exactly what it did, though I do know the general gist. It uses CoC (circle of confusion… Confusion) to blur parts that are not in focus. However, it seems that some adjustments have to be made, either to my focus algorithm or to the blurring logic. Most probably, I need to fix both. This is definitely something I need to look into… This is nowhere near perfect.

Focus sweeping and creating a depth map now work successfully! I have stored the frames from each focus sweep in cache rather than storing them in memory, as storing them in memory caused the app to crash on some devices. The sweeps are then processed and stored in physical, non-cache storage along with the original image. I have also been able to create a depth map that shows the depth in an image. However, the depth map will need calibration, as the algorithm does not perform well in low light (it uses SML, but it does not seem to remove artifacts well, and blank areas are a problem). Since I am running out of time (1 day left), I will focus on creating the blur using the depth map and the original picture. I did use AI to make parts of this (mainly on how to extract the Y plane, save it, then read it, as that was confusing).

I am currently attempting to take individual images for depth. I am using AI to help me here, along with previous repos, as the CameraX and Camera2 APIs are not very extensive or well-written and are very confusing to use. I am also running into many problems in my code at the moment. My code is crashing due to the amount of memory used, and the camera preview is not the correct size anymore. I have also implemented a shutter button to allow me to capture images.

I have now implemented tap-to-focus in my preview! Implementing this came with many bugs, mainly in handling tap-to-focus on fixed-focus devices and in creating the fade-in/out animation. There were also a couple of bugs in the preview that were much harder to fix, which varied across phones (I am testing on the Samsung S24 Ultra, Pixel 10 Pro XL, and Pixel 7 Pro, with the Pixel phones being the primary ones). These bugs were mainly the screen disappearing occasionally and not being able to click on a Logical Camera to see the underlying physical camera when there were no Physical Cameras within the Logical Camera.

I have now added the camera preview! This proved to be much harder to do than it originally seemed. In order to show the preview, I had to switch to using CameraX API instead of using Camera2. This was because I constantly ran into glitches where the preview was stretched, or rotating the preview would cause problems. I have also made the orientation always portrait when in the camera preview. This helps the fluidity of the UI as you will not see the screen tilting animation when turning the phone (usually, camera software only tilts the icons or buttons). At the moment, the preview only shows the camera feed. I also used AI to help me with the logic for the camera preview and changing my project from Camera2 to CameraX.

All previous glitches have been fixed. I had a problem where one Logical button would cause all buttons to show their Physical buttons. I also fixed other possible edge cases I overlooked that may cause my app to use more memory than it needs. I also slightly redesigned my buttons after realizing that Cards can be clickable. The different colour in the screenshot is simply because I changed my phone’s Material You theme (all colours are based on your phone’s theme by default). Small changes such as uneven padding for main cards and smaller padding for smaller cards helps the viewer group the cards. Rounding numbers for readability and a change in title have also been added.

Ran into some problems… Thought I got the view and logic right for the buttons, but it seems I did not.

Time to redo the button styling and logic! (I’m fried)

Finished the basis for my project. Time to start working on the camera preview and handling the physical lenses separately.