The domain https://fediverse.observer/ redirects to sedo now.
On October 11th these two commits where made to the repo:
this will be my final release, glad to have contributed since 2011 to some fediverse stats, https://fedidb.com/ is a great resource with more interest
this software is no longer developed
Sad to have lost such a great resource.
Yeah. I saw the dead website earlier today and have been trying to figure out how to keep the posts working.
The fediverse software post, I could rewrite some code to gather fedidb daily (to my knowledge, they don’t have all time series data available), but their MAU math is suspect. So, not a great option, but my only other thought is basically remaking fediverse-observer and I do not have a server or potentially relevant server skill set beyond cron jobs.
For the threadiverse instances post, I can keep gathering data from the instances I already have in my database, but don’t really have another way to find new ones. Maybe, I could see if I could scrape it from fedidb (I couldn’t get their API working in the past).
In conclusion, the posts are not in a good place :(
Oof the stats shown in your last post there make the Threadiverse look very unhealthy.
For one, the total Lemmy MAUs falling down to the <30k range (at the high mark during Rexodus it hit ~65K, and we held steady at ~40-45K for a long time it seemed, iirc).
And for another, if I am interpreting the HYAU correctly then literally thousands of people who had accounts here are simply walking away and never coming back (then hundreds for PieFed too, though still net negative). This is crucial bc these are not people who were too daunted by the complexity of learning about the Threadiverse, picking an instance, making an account on it, and joining communities - they were already here, and now they are not here anymore.
Perhaps they are avoiding coming here (even once) during the upcoming season of Western elections, with all the contentious content that is already flooding this place leading up those, but then they might come back later? Given past trends, I doubt it, though anything is possible.
The Threadiverse looks to be slowly shrinking. Perhaps the existing people on it post more often, but with an ever-diminishing pool of recipients for that content. This will lead to a feed-forward cycle where even fewer will want to join.
And to be clear I don’t blame those people who left in the slightest - I find myself visiting here far less often than I used to as well. There’s only so much politics that I want to argue about before I find myself missing the ability to discourse on some other, any other, topic.
I think I can alleviate some of your worries. The instance post only shows the top 20 instances for a software (Lemmy has 287 active, Piefed: 59, Mbin: 16).
My Fediverse Growth Post is a better indicator for the Threadiverse’s health. From there you can see the Threadiverse is only down 2.1% from a year ago (-847 MAU). From what I’ve seen over the last >6 months, is that the Threadiverse, and most of the fediverse, grow in short spurts and then slowly declines until the next spurt.
So, unfortunately, we aren’t having continuous positive growth, but I’d say we are stable. I think stable is fine as all the fediverse developers are still adding functionality/interconnected-ness (‘this isn’t our final form’).
For HYAU (Half-Year Active Users), I think it’s important to remember that lots of times people move instances, so their old account will eventually stop being counted as active (I personally have 2 old accounts). From the Fediverse post, you can see there were 42889 MAU 6-months ago and 42276 MAU 3-months ago (-613 MAU). So, even though you may see >1000 HYAU drop off in a given week, the total number of active users over that next 3 months stayed relatively the same (-1.4%, not -28% (-1000 MAU x 12 weeks)).
I think the most important information in the instance post is MAU/HYAU (%) (“Stickiness”), which indicates how well an instance holds onto its users (they keep coming back). I think >60% should be the goal of every instance (67% is currently the peak for non-small instances (>30 MAU)).
It might be worth a study to figure out why there is such a discrepancy across instances. For example, is it correlated to UI issues/customizability, reliability/up-time/longevity, admin/mod behavior, instance rules, federation/de-federation, inter-instance activity (communities/posts/comments), etc.
Regarding politics, I can’t really say what an average or new user experiences. I’ve curated/blocked massive amounts of political content. It might be worth having an instance that initially auto-blocks all politics communities for a new user. I don’t recall exactly, but I think Tesseract? was like that, but the dev/admin was sketchy about what they were doing and then melted down when called out for hiding things. So, it seems the idea had some popularity, but needs/ed a better implementation.
Thank you for your detailed response.
So the “Sum” mentioned at the bottom of each section is only for those shown within the section - the top 20 - and not expanded for all of the category? e.g. Lemmy’s MAU went down from 30817 to recently 29683, and every other metric (even for the most part the other post you mentioned for Fediverse Growth) also shows decline, but you are saying that despite that, the numerical amount is not that much, and we are mostly stable?
That is very good to hear:-).
Here is an example discussion of a community asking its members if it wants to migrate to Lemmy: https://www.reddit.com/r/fossdroid/comments/1uiywal/should_we_switch_to_lemmy/. A few examples that seem representative to me:
Here’s another example from one month ago: https://www.reddit.com/r/MorpheApp/comments/1wdbscs/we_should_switch_to_something_else/
Or here’s a third example, from 2 months ago: https://www.reddit.com/r/oldreddit/comments/1vvylak/thoughts_on_lemmy/
Opinions there run the gamut from useless due to no content to tribal politicking scaring them away from the entire federated model, though some people like it, yet don’t come here due to too few features in comparison to Reddit (with UBlock Origin tweaks to keeps some of its more insane issues in check).
We here, who have the early adopter mindset, are willing to put in the time and effort to learn how and what to block. But e.g. the most popular 3rd-party app Voyager does not - by default - show the instance name of each user account, so a new joiner won’t be able to easily distinguish when e.g. someone from the tankie triad is trolling them vs. someone attempting to converse in good conscience. Wish and hope as we might - without putting in any actual efforts to change anything mind you - we simply cannot get away from the https://en.wikipedia.org/wiki/Paradox_of_tolerance principle.
We are a tankie / troll bar, and the vast majority of mainstream non-technical normal folks tell other people that they very much do not enjoy visiting here as a result. Admiral Patrick very much had a bad implementation to a somewhat good idea: being forced to view toxic content should be an option that users have some control over, somehow, even if that choice is disabled by default. e.g., it would be nice if at least one singular major instance would defederate from lemmy.ml, or in the case of a PieFed instance apply an instance block by default - but sadly, literally none of them do. Of course, they would simply use their alts to get around any such block, but otherwise we are back to the Paradox of Intolerance again: with next to zero distinction made between trolls vs. non-trolls, why would most people want to come here to discuss things with us in our troll bar?
No problem. Happy to provide insight into my workings :)
Yes and yes.
Yeah, the snippets and links you provided match my understanding of the situation. I don’t recall who or where, but I recall someone saying they were having a much easier time recruiting people to Piefed rather than Lemmy, due to Lemmy already having an established reputation. That being said, I expect the Lemmy 1.0 launch will cause a significant bump.
I found the user saying that, but they didn’t elaborate on what missing features they wanted. It’s been years since I used Reddit, do you have any clue what features they may be wanting?
I recall running into this issue and being confused, before I found the setting. Do you know if this is still the default? If so, and the Dev hasn’t given a specific reason for the decision (I can’t immediately find a post about it), I don’t mind making a post to ask them to switch the default.
Agreed.
Probably, but I think it would be an order of magnitude fewer users encountered, than if the block didn’t exist. If they did use alts to get around the block, their alt’s instance’s admin could be contacted. My understanding is that the “troll behavior” on .ml is allowed, may be even encouraged, by their admins/devs. So, different admin may not have a tolerant attitude toward it. Along with this, as more communities spread out from the flagship and most popular instances (.ml, .world, .social), I think things will get less toxic too.
As an aside, over the last 3-months, it looks like most of the “politically” focused instances are the ones on the heavier declining side of the range. The instances declining the least or even growing are on the “unaligned” side of things.
Man that sucks. But also because I never have before, I want to thank you for all your labor making those posts! I find them very interesting.
Glad you enjoy them :)
Just need to figure out how to keep them going properly. I have a couple thoughts I’ll look into tomorrow. Hopefully one of them bears fruit.
Good luck!
Do they do any math? I assumed it’s simply additive. They get the MAU from individual instances via their crawler, then display a total?
Differences to other sites is probably due to the set of instances checked, not anything math related.
https://fedidb.com/software/lemmy?search=lemmy.world&stats=1
Fedidb doesn’t seem to include lemmy.world, which seems like a substantial omission, as I believe that it’s the largest Lemmy instance.
EDIT: And my own home instance, lemmy.today, also doesn’t seem to be on there:
https://fedidb.com/software/lemmy?search=lemmy.today
The largest user count server that they seem to query is lemmy.ml:
https://fedidb.com/software/lemmy?search=lemmy.ml
EDIT2: Your home instance, kbin.earth, does seem to be on there:
https://fedidb.com/servers/kbin.earth
That is interesting to know. It doesn’t look like lemmy.world has opted out, judging by their robots.txt.
Oh, that’s a thought. I didn’t think of checking robots.txt.
Lemmy.today does have a blanket ban on all bots in its robots.txt, which I assume was put into place to help try to reduce (sporadically severe) scraping load. I’m not sure that the admin, @mrmanager@lemmy.today, realized that it might have cut it out of Lemmy statistics gathering.
Yeah I know, but its worth it. :)
I knew to check robots.txt because of fedidb’s Data Sources page, which specifies what data they get, their user agent, and how to opt-out via robots.txt.
One thing I’m noticing in regards to lemmy.world, I wonder if they ever disallowed access to the nodeinfo endpoint in their robots.txt in the past. They don’t currently, but the wording suggests that having that rule at any point will forever remove the instance from fedidb. Which could explain what we’re seeing for that instance.
You’d think it’s straightforward and additive. I added up all their listed instances for a couple different softwares. The math was wrong and overestimated the MAU count. I checked it a few times across a couple weeks. Couldn’t quickly figure out a pattern to how the math was wrong each time.
Maybe they fixed it in the last ~6 months, but I wouldn’t count on it. I believe it’s the same Dev as pixelfed and loops. So, not sure they’ve done any work on fedidb in years (to my knowledge, they’ve focused on Loops and Pixelfed).
Edit: to clarify, adding the MAU of all their listed instances for a particular software was lower than the total MAU they reported for the software.
Edit 2: Just checked their Piefed listings and it’s still wrong. Fedidb states: ~7.0k, actual addition: 6858 MAU.
Have you contacted them about the discrepancy? Ideally FediDB could just become the new Fediverse Observer.
Missing out on accurate data about Fediverse growth would really suck. 😕
I just confirmed the problem still exists and messaged them via mastodon about it.
Probably didn’t let them know originally because I didn’t have a mastodon account at the time and already had a good option (fediverse-observer).