You are browsing as a guest. Sign up (or log in) to start making projects!

rag feed reader

By @keyboardspam

  • 11 Devlogs
  • 28 Total hours

RSS reader for Emacs. This is a feed reader for emacs that aims to be better than [elfeed](https://github.com/emacs-elfeed/elfeed) by focusing on performance to prevent xml parsing from freezing the Emacs LISP thread because I use [EXWM](https://github.com/emacs-exwm/exwm) (a window manager written in ELISP) so whenever Emacs freezes up, I cannot use my computer for anything else unless I switch TTYs. Due to this, I will be trying to make it as fast as possible.

Source
Open comments for this post

@keyboardspam on rag feed reader

1h 7m 16s logged

After figuring out how to use cargo build with makefiles to test elisp dynamic modules, and replacing it in my other project, I replaced the build system from a configure script + makefiles to makefiles + cargo to run tests.

0
0
1
Open comments for this post

@keyboardspam on rag feed reader

5h 40m 49s logged

I made parsing happen inside of a thread pool in order to have parallelism to not block the main thread. Everything now happens much faster and is more responsive now.

0
0
5
Open comments for this post

@keyboardspam on rag feed reader

3h 1m 27s logged

The dublin core, syndication, content and media namespace parsers are now complete and all feed types are able to support them. Now you view feeds that don’t use most of their native elements.

0
0
2
Open comments for this post

@keyboardspam on rag feed reader

3h 50m 46s logged

I added support for the Dublin Core (dc) and Media RSS (media) namespaces in feeds. My favorite part of this was handling date and time because it is always just so fun. In Dublin Core, not only do the specs say that you can have incomplete ISO-8601 timestamps like (2000), (2000-01) but you could also specify time ranges like (2000/2000-02-01) https://www.dublincore.org/specifications/dublin-core/dcmi-terms/#date meaning the publishing date is somewhere between 2000-01-01 and 2000-02-01 leading to this amazing function. The best part of all this is that there is going to be more complicated date time handling when I write the syndication (sy) namespace handler.

0
0
4
Ship #1 Pending review

@keyboardspam on rag feed reader

This project is a feed reader for emacs that focuses on performance.

The feed reader that I used before this is [elfeed](https://github.com/emacs-elfeed/elfeed) so I will be comparing this with that.

QOL Improvements (over elfeed)

1. Memory usage
The elfeed implementation does a bunch of things that hog memory. For one, the database is held entirely in memory. Rag uses a sqlite database to consume less memory. Another thing is that the xml data is parsed into an AST before it gets processed by the parsers which means that you have both the original XML text and an AST which is duplicated work. To fix this, in rag it uses a xml streaming library (quick-xml) to process xml without an AST.

2. Performance
The biggest focus of this is in performance. One of the things that is improved over elfeed is that the elfeed buffer gets redrawn after every update by erasing the buffer and then redrawing everything which is very inefficient. This consumes a lot of time when elfeed finishes parsing feeds now that the updates are incremental and lazy, we are already multitudes faster than elfeed. There are more optimizations such as that are implemented using arena allocators to reuse allocations, or using references to avoid copying data.

The reason that I need my feed reader (or anything else I use in Emacs) to be very fast because on Linux (I use an Arch derivative btw) I use EXWM as my window manager so whenever ELISP code blocks, I cannot use my computer to do anything else (like remembering to use Emacs as a text editor). As a result I want my feed reader to be as fast as possible.

3. Reliability
One of the issues that I have with elfeed is a bug (that I haven't reported because idk how to describe or reproduce, I also don't have any screenshots) where after the elfeed database gets large enough with entries, the description of entries ends up incorrect where it cuts off halfway or it shows you the description of the wrong entry. I suspect that this has something to do with the fact that the database they use is a custom one implemented in ELISP (before Emacs got sqlite support) so by making a feed reader that uses sqlite which is very battle tested, in my limited experience so far, the database is more reliable here than in elfeed.

Try project → See source code →
Open comments for this post

@keyboardspam on rag feed reader

4h 4m 44s logged

The entries that are parsed from the feeds now get inserted into the buffer using binary search to incrementally you new entries. All that would be left is to create a guide for non-Emacs users so that people can tests it and also CI builds for releases for a bunch of different architectures then I could ship.

0
0
2
Open comments for this post

@keyboardspam on rag feed reader

1h 19m 12s logged

I added an entry viewer so now you can look see the description of feeds. The ui is “inspired” by elfeed. The only thing left before shipping would be a configuration interface for feeds to add your rss feeds followed by a scheduler for downloading them.

0
0
4

Followers

Loading…