Zos_Kia

Ce qu’il faut réaliser c’est que Google et Facebook c’est trivial à répliquer du point de vue technique. Si d’un coup on est coupés de ces services on peut en lever 12 alternatives en une semaine.

Par exemple les alternatives à Workspace fleurissent en ce moment, entre la Suite Numérique (pour la fonction publique), Twake, la kSuite d’Infomaniak etc…

Je pense que ce qu’on devrait se demander c’est plutôt comment l’Europe peut faire pour ne pas créer un FB ou un GOOG européen. On n’a pas forcément besoin de ces mentalités de dragons qui veulent tout dévorer. Un écosystème de services interopérables c’est 100x plus désirable que 3 acteurs monopolistiques.

Yes i saw that benchmark and was honestly not surprised with the results. It seems that Anthropic really focused on those issues above and beyond what was done in other labs.

Je suis pas un cypherpunk OG mais je pense que le problème avec les blockchains c’est qu’elles ne disparaissent jamais. Tu as des boites de data mining avec des pré-requis de zinzin et des dizaines de milliers de GPU qui passent 365 jours par an à taper dedans, et ce qui n’est pas possible aujourd’hui peut le devenir demain.

Comme tu disais, on est plus à l’ère de Bitcoin, justement on y est plus parceque le data mining a énormément progressé et qu’il est trivial de dé-tumbler 1 million de transactions par secondes, de clusteriser les portefeuilles et de faire d’autres dingueries dont j’arrive pas à prononcer le nom… C’est un des rares cas de figure où l’enquêteur a l’avantage de l’attaquant, il lui suffit d’avoir un coup de bol une fois pour établir une corrélation.

Après ça invalide pas ton commentaire sur l’usage légitime mais je pense qu’il est bon de se rappeler que l’anonymisation c’est un état temporaire. Si c’est sur 100 ou 100K ans c’est bon pour toi, si c’est sur 12 ans c’est moins bon, ça dépend des usages 🤷

Honestly Claude is not that sycophantic. It often tells me I’m flat out wrong, and it generally challenges a lot of my decisions on projects. One thing I’ve also noticed on 4.6 is how often it will tell me “I don’t have the answer in my training data” and offer to do a web search rather than hallucinating an answer.

Interestingly, Roots was inspired by the first nu-metal albums, especially Korn’s first album. That’s crazy to me and i would have guessed it was the other way around.

I think a lot of it is still done by hand, and there is also synthetic data distilled from larger models of course.

I mean, making fuel is one thing

Actually i was more thinking of crude metallurgy and materials processing. You could quite easily get aluminium from lunar regolith, and also tons of silicates. This allows you to produce shielding, radiators and the structural elements of solar panels without having to kaboom-boom the tons of raw material from the Earth. And it’s not particularly high-tech stuff either, just some furnaces and basic extruding would go a long way. If you just have to ship the delicate electronics from Earth you’re already saving a lot.

Il m’est arrivé exactement la même chose il y a un an à peu près. L’idée c’est qu’ils te font poser l’argent sur leur wallet en te faisant croire que tu es en train de créer le tien. C’est assez subtil parceque chaque étape est crédible, oui pourquoi pas déposer les coins en self-custody le temps de reset le compte c’est pas hyper déconnant en soi si t’y connais superficiellement en cryptos. Mais il y a un moment où il t’embrouille pour te faire cliquer le mauvais bouton. Au lieu de te faire créer un wallet (et recevoir ta ta liste de pass phrases), il te fait importer un wallet. A ce moment exact il t’envoie un faux mail de Binance avec une liste de pass phrase (qui est en fait celle de son wallet) donc tout est cohérent, dans l’app ça te demande 12 mots dans le mail il y a 12 mots tu copies sans trop te demander. Une fois que le wallet est dans ton compte tout ce que tu transfères dedans est à eux.

Dans ma chance, j’ai juste transféré 1€60 de shitcoin avant de sortir de l’hypnose et commencer à résister. A ce stade le gars était trop excité je pense qu’il se voyait si près du but. Dès que j’ai mis une objection il s’est frustré très vite et là c’était grillé.

Interestingly NASA had an idea of a plan that sounds at least technically possible, but it’s a multi-decade operation and doesn’t look anything like what the current startups are pitching. Of course you can have your data centers in space, why the fuck not, but a data center sits on top of a lot of boring old infrastructure which nobody’s excited to talk about.

It’s going to be prohibitive if you have to pay the gravity tax every time you want to move 1 ton of metal, so realistically this kind of high-tech project cannot even begin without having substantially industrialized the moon. Nothing fancy but you’ll need at least some mining and refining, and solid trans-lunar logistics routes. Probably some housing for a bit of personnel too. At that point the space data center would be dwarfed by the size of its own support system.

The best way to learn to write is to write and have someone critique you. That someone can be an AI it doesn’t change anything about the process, as long as the initial input is your own best effort and the final result is your own edit based on the feedback you received.

That’s an excellent point! On that topic I recently listened to an interview of the founder of EleutherAI, who focuses on training small language models. She said they were able to train a 1B parameters reasoning model with 50K Wikipedia articles and carefully curated RL traces. The thing could run in your smartphone and is at parity with much larger models trained on trillions of tokens.

She also scoffed at Common Crawl and said it contained mostly cookies and porn. She had a kind of attitude like “no wonder the big labs need to slurp trillions of tokens when the tokens are such low quality”. Very interesting approach, if you understand french I can only recommend the interview.

OK that’s a fair observation. Honestly my naive guess would be that they simply do not optimize mainline gpt models for the kind of use case you generally have on Api (tool use, multi-step actions, etc…). They need it to be a perky every day assistant not necessarily a reliable worker. Already on gpt-4 i found it extremely mediocre compared to the Claude models of the same time.

I think that’s a more likely explanation than model collapse which is a really drastic phenomenon. A collapsed model will not just fail tasks at a higher rate, it will spit garbled text and go completely off the rails, which would be way more noticeable. It would also be weird that Claude models keep getting better and better while they’re probably fed roughly the same diet of synthetic data.

The switch you mention (from 4th gen to 5th gen GPT) is when they introduced the model router, which created a lot of friction. Basically this will try to answer your question with as cheap a model as possible, so most of the time you won’t be using flagship 5.2 but a 5.2-mini or 5.2-tiny which are seriously dumber. This is done to save money of course, and the only way to guarantee pure 5.2 usage is to go through the API where you pay for every token.

There’s also a ton of affect and personal bias. Humans are notoriously bad at evaluating others intelligence, and this is especially true of chatbots which try to mimic specific personalities that may or may not mesh well with your own. For example, OpenAI’s signature “salesman & bootlicker” personality is grating to me and i consistently think it’s stupider than it is. I’ve even done a bit of double blind evaluation on various cognitive tasks to confirm my impression but the data really didn’t agree with me. It’s smart, roughly as smart as other models of its generation, but it’s just fucking insufferable. It’s like i see Sam Altman’s shit eating grin each time i read a word from ChatGPT, that’s why i stopped using it. That’s a property of me, the human, not GPT, the machine.

I’m sorry but no, models are definitely not collapsing. They still have a million issues and are subject to a variety of local optima, but they are not collapsing in any way. It is not known whether this can even happen in large models, and if it can it would require months of active effort to generate the toxic data and fine-tune models on that data. Nobody is gonna spend that kind of money to shoot themselves in the foot.

Yeah i remember that Ed article ! I don’t think the technical aspects are relevant to the newer generation of models, but yeah of course any attempt to compress inference costs can have side effects : either response quality will degrade for using dumber models, or you’ll have re-inference costs when the dumb model shits its pants. In fact the re-inference can become super costly as dumber models tend to get lost in reasoning loops more easily.

Yeah that’s also something that you have to train for, i’m not super aware of the technicals but model routing is definitely important to the AI companies. I suspect that’s part of why they can pretend that “inference is profitable” as they are already trying to squeeze it down as much as possible.

I mean what’s the point of morging if you’re gonna do it intermittoucly?

To clarify : model collapse is a hypothetical phenomenon that has only been observed in toy models under extreme circumstances. This is not related in any way to what is happening at OpenAI.

OpenAI made a bunch of choices in their product design which basically boil down to “what if we used a cheaper, dumber model to reply to you once in a while”.

I don’t know about other bands but the bit about iron maiden is really stretched.

I guess if you consider the first lineup to be the one for their first concert in a bar’s basement, alright. But if you take the first album, Dave Murray was already in the band and still is.

testing is doubting