Dessalines

Dessalines@lemmy.ml · 4 days

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Changes

This release significantly reduces memory usage for the Lemmy backend. Metrics on production instances show a reduction up to 10 times. Here you can see the statistics from a few different instances. Read on for a technical explanation below. Note that the change only affects x86, there is no difference on ARM (e.g. Raspberry Pi), see here for details.

lemmy.ml (overall RAM usage including OS, Docker, PostgreSQL etc):

leminal.space (backend container only)

lemmy.world (backend API containers)

lemmy.world (backend federation and scheduled tasks)

How was such a major improvement possible? Memory usage in languages like Rust or C is managed by a so-called memory allocator. It requests large chunks of RAM from the operating system, and provides smaller chunks when needed in the program. For example, every string (such as post titles or markdown text) require chunks of memory to store them. When processing the data is completed, the memory should be freed and released, or reused.

Until version 0.19.19 Lemmy used the mimalloc memory allocator, which is supposedly better than the default glibc allocator. Yet a recent blog post points out that mimalloc does not play well with the tokio async runtime. It suggests to use jemalloc instead. Changing memory allocators is very simple in Rust, it only requires a single line of code. So we tried that, deployed the change on lemmy.ml and it immediately proved effective.

Upgrade instructions

There are no breaking changes with this release.

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Thanks to everyone

We’d like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We’re glad many people find it useful and enjoyable enough to contribute.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over five years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. A recurring donation is the best way to ensure that open-source software like Lemmy can stay independent and alive, and helps us grow our little developer co-op to support more full-time developers.

Dessalines@lemmy.ml · 7 days

You can’t simply say “we must assume” as evidence.

Okay yeah you definitely didn’t read it. Large sections in that doc just before that are on phone number identifiers, NSLs, and 5-eyes countries, the US goverment pushing signal in privacy spaces… literally the reasons why signal isn’t trustworthy. Unless you can tell me what an NSL is, then I’ll assume you didn’t read it.

While there may be other US government requests they are not alllwed to disclose, they were legally required to provide the same information to the courts, and we can see what they provided.

Did you ignore the large section on NSLs? These come with a gag order, meaning its illegal for signal to notify their users about them being spied on.

In fact, they implemented “Sealed sender” in 2018 where they are not able to see who the message is being sent to.

This is a “just trust me” from signal, since neither of us have access to their centralized DB, but you also ignored two paragraphs down, where it showed that with message timestamps and recipient information, this would be trivial to find the real sender of a message, regardless of sealed sender. Again, actually open source software can’t say “just trust me” like signal can, we actually have to show code to prove it, and let people run that code in a private manner.

And sure, while the US government funds Signal, you know who else endorses it? Edward fucking Snowden. If anyone knows about secure messaging, it’s the man that physically removes the microphone and camera from his phones before using them.

Elon musk and jack dorsey also endorse signal. An endorsement means nothing, especially for centralized software based in a 5-eyes country.

Dessalines@lemmy.ml · 7 days

Read the linked doc, because it’s clear you didn’t.

Dessalines@lemmy.ml · 7 days

Signal is a centralized, US-based service which requires your phone number (thus your real identity, IE name and address), has social networking graphs of everyone you talk to, and must forward that information to the US government when asked, as well as (by law) not tell you that they’ve been asked to do so. During the Obama era, 60 NSLs were issued for this private information every single day.

People overlook its privacy concerns for the same reason they do with apple: it has a shiny interface and is easy to use, and makes people very attached to it. Behind all that, is a surveillance network that its creators have explicitly said they do not want it to be able to run in a decentralized, private manner.

It has a long history of privacy offenses below (such as refusing to publish its server’s source code for years, its reliance on other US tech services (amazon, google), US-government funding, and a US-defense-tank friendly administration) which get ignored or shouted down by many of those above. See the article below.

Why not signal.

Pretty much any alternative is better, as long as its not hosted in a five-eyes country, and especially if it doesn’t require phone numbers or real identities like signal does.

I personally have been using SimpleX for friends and real life contacts, and Matrix for larger more anonymous group chats.

Dessalines@lemmy.ml · 5 months

The charts above show growth of active users on thelemmy.club (left) and lemmus.org which are suggested on join-lemmy.org. It shows that improvements to the official project site really have a (small) influence on user growth. There are also a few new features:

This past month we’ve been working on lemmy-ui bug fixes and visual improvements, and a few remaining lemmy back end issues to get ready for the 1.0 release. We’ve closed over 20 lemmy-ui milestone issues, and have only a few remaining. The only major issue left is performance-testing, to ensure all our database changes and optimizations are working correctly.

You can follow our progress with these milestone links:

If you’d like to see the new features and visual updates in production, you can visit voyager.lemmy.ml which deploys updates every night. Please help by testing the new features and reporting any problems.

If you have any experience with web development or want to learn it, consider contributing to lemmy-ui. It is written in standard Typescript with Bootstrap.

If you have experience with Kotlin or Android development, you can help contribute to lemmy’s open source android app, Jerboa.

Thank you to everyone who has helped out with testing, development, spreading the word about lemmy, and building communities. Your help has brought lemmy from an idea to one of the most vital pieces of software in the fediverse.

Here are the major changes from February:

Full list of changes by user

malsadev

Support translations in RSS feeds (fixes #5365)

iByteABit256

Added native support for warnings

flamingos-cant

dessalines

Nutomic

Or see the full list of changes at the links below:

An open source project the size of Lemmy needs constant work to manage the project, implement new features and fix bugs. Dessalines and Nutomic work full-time on these tasks and more. As there is no advertising or tracking, all of our work is funded through donations. Even so there is barely enough time in the day, and no time for a second job. The only available option are user donations.

To keep it viable donations need to reach a minimum of 5000â‚¬ per month, resulting in a modest salary of 2500â‚¬ per developer. If that goal is reached we can stop worrying about money, and fully focus on improving the software for the benefit of all users and instances. We especially rely on recurring donations to secure the long-term development and make Lemmy the best it can be.

Donate

Dessalines@lemmy.ml · 11 months

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Changes

This version fixes a handful of bugs. Most importantly it prevents a background task from blocking the Lemmy server entirely. Edit marks on comments are now hidden if it was edited within 5 minutes of creation. Also the browser cache usage is significantly reduced. Additionally ARM builds are fixed by using the default memory allocator on that platform.

Backend

Fixing active counts slow queries. by @dessalines in #5907
Only use mimalloc on x86 and purge images in background task by @Nutomic in #5893
Add missing comment depth check by @Nutomic in #5842
Dont sanitize RSS content manually (fixes #5850) by @Nutomic in #5852
Reduce false positives in URL blocklist to reduce scunthorpe problem by @Nothing4You in #5807

Frontend

Don’t show edit mark if comment was edited in less than 5 minutes by @jfaustino #3197
Increase bio max length to 1000 chars by @nutomic #3249
Change link from element.io to matrix.org by @nutomic #3250
Remove all caches (fixes #3195) by @Nutomic in #3248
Fixed ordering for search results by @Nutomic in #3219
Add search field to community sidebar by @Nutomic in #3217
Add checkbox for title only search by @Nutomic in #3220

Upgrade instructions

There are no breaking changes with this release.

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Thanks to everyone

We’d like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We’re glad many people find it useful and enjoyable enough to contribute.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over five years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. A recurring donation is the best way to ensure that open-source software like Lemmy can stay independent and alive, and helps us grow our little developer co-op to support more full-time developers.

Dessalines@lemmy.ml · 2 years

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Changes

This release took a long time to complete due to a major performance problem which brought lemmy.ml to a crawl every time we tried to deploy the new version. It took a lot of testing (in production) to narrow it down to a single commit, and finally fix the problem.

The release itself contains numerous bug fixes and minor improvements:

Lemmy

Enhancements

Parallel federation sending by @phiresky in #4623
Reduce CPU usage for generating link previews by @phiresky in #4957
Switch from OpenSSL to rustls by @kwaa in #4901
Increase max post url length to 2000 characters by @dessalines in #4960
Increase max length of user bio to 1000 charactes by @dessalines #5014
Reduce maximum comment depth to 50 by @nutomic #5009
Resize post thumbnails by @nutomic #5107/files
Add category to RSS feeds by @nutomic #5030
Allow users to view their own removed/deleted communities by @dessalines in #4912
Add backend check to enforce hierarchy of admins and mods by @dessalines in #4860
Do pictrs transformations for proxied image urls by @dessalines in #4895
Enable more build optimizations by @nutomic in #5168
Calculate “controversial” ranking with exponent instead of multiply (just like Reddit) by @dullbananas in #4872
Automatically remove tracking parameters from URLs by @dessalines #5018
Relax timeout for sending activities by @Nothing4You in #4864

Bug Fixes

Fix admin notification for new user registration (fixes #4916) by @Nutomic in #4925
Allow community settings changes by remote mods @flamingo-cant-draw in #4937
Fix problem with connecting to Postgres with TLS @FenrirUnbound in #4910
Fix bug when commenting in local-only community by @dessalines in #4854 and @abdel-m in #4920
Fix scheduled task to delete users with denied applications by @Nothing4You in #4907

API

Return image dimensions and content type in API responses by @dessalines in #4704
Adding a show_read override to GetPosts. by @dessalines in #4846
Add show_nsfw override filter to GetPosts. by @dessalines in #4889
Require authentication for site metadata fetch endpoint by @dessalines in #4968
Add the ability to fetch a registration application by person_id by @dessalines in #4913
Order community posts by published data, not id by @dullbananas in #4859
Throw error when non-mod posts to mod-only comm or when URL is blocked by @flamingo-cant-draw in #4966
Add option to search exclusively by post title by Carlos-Cabello #5015

Database

Approve applications in transaction by @Nothing4You in #4970
Use trigger to generate apub URL in insert instead of update, and fix query planner options not being set when TLS is disabled by @dullbananas in #4797

Lemmy-UI

Fix full-size post images. by @dessalines in #2797
Fix modlog ID filtering. by @dessalines in #2795
Allow Arabic and Cyrillic characters when signing up or creating community by @SleeplessOne1917
UX - Swap “Select Language” and “Cancel/Preview/Reply” button locations around in commentsReverse order of buttons in Reply TextArea
Fix jump to content by @SleeplessOne1917
Fixing peertube and ordinary video embeds. by @dessalines in #2676
Changing sameSite cookie from Strict to Lax. by @dessalines in #2677
Remove show new post notifs setting. by @dessalines in #2675
Fix memory leak around emojis on server render by @makotech222 in #2674
Enable spellcheck for markdown text area by @SleeplessOne1917 in #2669
Pre release dep bump by @SleeplessOne1917 in #2661
Add ability to fill magnet link title on post creation. by @dessalines in #2654
Registration application view by @SleeplessOne1917 in #2651
Add torrent help by @dessalines in #2650
More moderation history by @dessalines in #2649
Fix tribute related bug by @SleeplessOne1917 in #2647
Remove min and max length from password input when using login form by @SleeplessOne1917 in #2643
Remove trending communities card from home. by @dessalines in #2639
Set data-bs-theme based on the presence of “dark” in theme name by @SleeplessOne1917 in #2638
Fixing modlog filtering to allow admins and mods to filter by mod. by @dessalines in #2629
Fix issue from logo bugfix by @SleeplessOne1917 in #2620
Make more post params cross-postable by @SleeplessOne1917 in #2621
Fix wonky comment action icon button alignment by @SleeplessOne1917 in #2622
Prevent broken logo from crashing site by @SleeplessOne1917 in #2619
Add rate limit info message. by @dessalines in #2563
Fix getQueryString by @matc-pub in #2558

New Contributors

@abdel-m made their first contribution in #4920
@johnspurlock made their first contribution in #4917
@FenrirUnbound made their first contribution in #4910
@kwaa made their first contribution in #4901
@Daniel15 made their first contribution in #4892

Full Changelog

Upgrade instructions

This upgrade could take as long as ~30 minutes for larger servers, due to needing to recalculate controversy ranks for all historical posts.

There are no breaking changes with this release.

Follow the upgrade instructions for ansible or docker.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Thanks to everyone

We’d like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We’re glad many people find it useful and enjoyable enough to contribute.

Special shout out to @SleeplessOne1917, @phiresky, @dullbananas, @mv-gh, @Nothing4u, @asonix, @sunaurus, @flamingo-cant-draw, and @Freakazoid182 for their many code contributions and helpful insights.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over five years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. A recurring donation is the best way to ensure that open-source software like Lemmy can stay independent and alive, and helps us grow our little developer co-op to support more full-time developers.

Dessalines@lemmy.ml · 2 years

This is a chance for any users, admins, or developers to ask anything they’d like to myself, @nutomic@lemmy.ml , SleeplessOne , or @phiresky@lemmy.world about Lemmy, its future, and wider issues about the social media landscape today.

NLNet Funding

First of all some good news: We are currently applying for new funding from NLnet and have reached the second round. If it gets approved then @phiresky@lemmy.world and SleeplessOne will work on the paid milestones, while @dessalines and @nutomic will keep being funded by direct user donations. This will increase the number of paid Lemmy developers to four and allow for faster development.

You can see a preliminary draft for the milestones. This can give you a general idea what the development priorities will be over the next year or so. However the exact details will almost certainly change until the application process is finalized.

Development Update

@ismailkarsli added a community statistic for number of local subscribers.

@jmcharter added a view for denied Registration Applications.

@dullbananas made various improvements to database code, like batching insertions for better performance, SQL comments and support for backwards pagination.

@SleeplessOne1917 made a change that besides admins also allows community moderators to see who voted on posts. Additionally he made improvements to the 2FA modal and made it more obvious when a community is locked.

@nutomic completed the implementation of local only communities, which don’t federate and can only be seen by authenticated users. Additionally he finished the image proxy feature, which user IPs being exposed to external servers via embedded images. Admin purges of content are now federated. He also made a change which reduces the problem of instances being marked as dead.

@dessalines has been adding moderation abilities to Jerboa, including bans, locks, removes, featured posts, and vote viewing.

In other news there will soon be a security audit of the Lemmy federation code, thanks to Radically Open Security and NLnet.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

Dessalines@lemmy.ml · 3 years

This is an opportunity for any users, server admins, or interested third parties to ask anything they’d like to @nutomic@lemmy.ml and I about Lemmy. This includes its development and future, as well as wider issues relevant to the social media landscape today.

Note: This will be the thread tmrw, so you can use this thread to ask and vote on questions beforehand.

Original Announcement thread

Dessalines@lemmy.ml · 3 years

This is an opportunity for any users, server admins, or interested third parties to ask anything they’d like to @nutomic@lemmy.ml and I about Lemmy. This includes its development and future, as well as wider issues relevant to the social media landscape today.

Dessalines@lemmy.ml · 3 years

Rule #2 is possibly our most important one:

Be respectful. Everyone should feel welcome here.

Learn to disagree without being rude or disrespectful.

It can be difficult sometimes, since western social media thrives on collective outrage, and they knowingly ingrain this into us for years. But please do adhere to this rule, and it will make this place much more enjoyable.

We will not hesitate to issue temp bans (usually a day or two) for those who make everyone’s experience unpleasant.Hit the report button if you see this behavior.

Thanks!