Made skysync work with custom domain and SSL certificate. Updated security and UI a little. Updated docker files. Now ready to ship!

Added Recent and important page. Added folder soft deletion. Added ransomware detector(I hope). Added word files preview. Added sounds files preview.

I didnt focus on UI as I said in previous devlog, I decided to add all functionalities first and then adjust UI. So I improved security page. I also updated backend to be more secure. Two or three devlogs more and I finish functionalities, then i am going on one week holiday. I’m targeting at 15-18 August to deploy

Improved main page UI. Added security page. Added limits while creating ZIP. Updated settings page. Fixed UI bugs.Now I will focus on mobile view

Updated dashboard UI a little. Updated settings page. Added download checksum, session managment resumable file upload. Added Typescript syntax. Added c/c++/cuda hihjlighter and syntax.

Hello, world!

Recently, I’ve been working on optimizing CUDA kernels for my own LLM implementation. Most of them are now performing quite well, especially considering that I’m running everything on an RTX 3050 Laptop GPU with only 4 GB of VRAM.

There is still one kernel that keeps frustrating me: GQA attention. Its GPU utilization is much lower than the rest, and I’m not sure how to optimize it further.

Here are my current benchmark results:

RMSNorm

• Input: [512, 4096]
• Data type: FP16
• Execution time: 0.049 ms
• GPU throughput: 89%

SwiGLU

• Input: [512, 4096]
• Data type: FP16
• Execution time: 0.074 ms
• GPU throughput: 90%

Linear layer

• Input: [512, 4096]
• Weights: [4096, 4096]
• Output: [512, 4096]
• Data type: FP16
• Execution time: 1.04 ms
• GPU throughput: 60%

Note: The reported throughput is relatively low because this benchmark runs GEMM on a comparatively small input matrix. At this problem size, the GPU cannot reach the same utilization as it would with larger batches or longer sequences.

Rotary Position Embedding — RoPE

• Query: [1, 512, 32, 128]
• Key: [1, 512, 8, 128]
• Cosine and sine cache: [512, 64]
• Data type: FP16
• Execution time: 0.06 ms
• GPU throughput: 87%

Grouped Query Attention — GQA

• Query and output: [1, 512, 32, 128]
• Key and value: [1, 512, 8, 128]
• Data type: FP16
• Execution time: 3.32 ms
• GPU throughput: 10%

This is currently the main bottleneck. I have already applied tiling, online softmax, and shared-memory optimizations, but the utilization is still very low.

Embedding lookup

• Number of input tokens: 512
• Vocabulary size: 32,000
• Hidden size: 4,096
• Output: [512, 4096]
• Data type: FP16
• Execution time: 0.05 ms
• GPU throughput: 88%

Summary

Overall, I’m satisfied with the results, but the attention kernel still needs a lot of work. (I will probably use cuDNN). The hardest thing before me - backprop. But now I can play with kernels and model layers.

Hello, world!

Recently, I’ve been working on optimizing CUDA kernels for my own LLM implementation. Most of them are now performing quite well, especially considering that I’m running everything on an RTX 3050 Laptop GPU with only 4 GB of VRAM.

There is still one kernel that keeps frustrating me: GQA attention. Its GPU utilization is much lower than the rest, and I’m not sure how to optimize it further.

Here are my current benchmark results:

RMSNorm

• Input: [512, 4096]
• Data type: FP16
• Execution time: 0.049 ms
• GPU throughput: 89%

SwiGLU

• Input: [512, 4096]
• Data type: FP16
• Execution time: 0.074 ms
• GPU throughput: 90%

Linear layer

• Input: [512, 4096]
• Weights: [4096, 4096]
• Output: [512, 4096]
• Data type: FP16
• Execution time: 1.04 ms
• GPU throughput: 60%

Note: The reported throughput is relatively low because this benchmark runs GEMM on a comparatively small input matrix. At this problem size, the GPU cannot reach the same utilization as it would with larger batches or longer sequences.

Rotary Position Embedding — RoPE

• Query: [1, 512, 32, 128]
• Key: [1, 512, 8, 128]
• Cosine and sine cache: [512, 64]
• Data type: FP16
• Execution time: 0.06 ms
• GPU throughput: 87%

Grouped Query Attention — GQA

• Query and output: [1, 512, 32, 128]
• Key and value: [1, 512, 8, 128]
• Data type: FP16
• Execution time: 3.32 ms
• GPU throughput: 10%

This is currently the main bottleneck. I have already applied tiling, online softmax, and shared-memory optimizations, but the utilization is still very low.

Embedding lookup

• Number of input tokens: 512
• Vocabulary size: 32,000
• Hidden size: 4,096
• Output: [512, 4096]
• Data type: FP16
• Execution time: 0.05 ms
• GPU throughput: 88%

Summary

Overall, I’m satisfied with the results, but the attention kernel still needs a lot of work. (I will probably use cuDNN). The hardest thing before me - backprop. But now I can play with kernels and model layers.

Successfully dockerized whole aplication

Implemented RoPE kernel. All 10 tests passed. I also added in code documentation. Netx big stepp will be attention kernel.

With chatgpt help i optimized my tokenizer. 100MiB needs only 33.72s of training time. Batch encode throughput is around ~400MiB/s which is around 11.5x faster than OpenAI tiktoken(look at images). I have CPU AMD 5 5600H inside PC. I think that on AI data center my tokenizer could reach even about 4GiB/s ! My tokenizer is fast but it require relative big amount of ram. Nedded RAM is about 20x of train file size. When I finish my project I will train my model and benchmark my tokenizer on god tier machine. Wish me luck!

With chatgpt help i optimized my tokenizer. 100MiB needs only 33.72s of training time. Batch encode throughput is around ~400MiB/s which is around 11.5x faster than OpenAI tiktoken(look at images). I have CPU AMD 5 5600H inside PC. I think that on AI data center my tokenizer could reach even about 4GiB/s ! My tokenizer is fast but it require relative big amount of ram. Nedded RAM is about 20x of train file size. When I finish my project I will train my model and benchmark my tokenizer on god tier machine. Wish me luck!

Succesfully improved Tokenizer. Now 100MiB of text encode in around 1 minute. In previous devlog 8MiB nedded 22 seconds. So huge improvemnt

Optimized tokenizer - 100Mib require 21.8s training time. Encoding time 5.5MiB/s

Implemented RMSNorm, linear layer kernel. Implemented BPE Tokenizer - currently its slow.

Transparent planets bug fix

Successfully complied CUDA hello world

Updated typescript highlighter. Added better files mobility - folder tree view. Drag & drop files into folders. Pausable/Resumable file upload.

Added typescript highlight support. Now insted of loading placeholder images are blured. Improved UI/UX. Updated mobile support. Fixed bug and vulnelabirity

Updated UI/UX. Fixed a bug causing planet collisions.
Added readme, License, gh actions. Added charts and camera follow.