Vechev and his team found that the large language models that power advanced chatbots can accurately infer an alarming amount of personal information about users—including their race, location, occupation, and more—from conversations that appear innocuous.

  • abhibeckert@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    9 months ago

    Unfortunately, the larger models are way too big to run client-side.

    Memory isn’t that expensive… NVIDIA generally only gives you a lot of it if you also buy a huge amount of compute (which is expensive), but there are other hardware manufacturers (e.g. Apple) that offer lots of memory with a modest amount of computer power and they run these models with great performance on hardware that doesn’t break the bank.

    Now that there’s a mass market use case for a lot of memory with a modest amount of compute power, I expect other hardware manufacturers will catch up to Apple and ship offerings of their own.

    You’d have to be crazy to let Google store all your personal emails for all eternity! And yet everybody does it

    There are other email providers…

    • GenderNeutralBro@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      1
      ·
      8 months ago

      Yes, I am impressed with Apple Silicon, and the fact that I can get up to 192GB of integrated memory in a laptop is pretty impressive. Unfortunately, the costs are still crazy high (Apple and overpriced RAM: name a more iconic duo), and even 192GB is only about half of what is needed for the largest models I’m aware of (e.g. BLOOM 366B). I don’t think OpenAI has officially stated how big GPT4 is, but it’s likely even bigger.

      The industry has been stagnating for a long time now in terms of memory, and I hope this will push prices down and capacity up.

      The good news is that there is strong motivation for companies like Apple and Google to shift more processing onto client devices, because the cost of running these servers is freakin’ huge.

      There are other email providers…

      You’re right of course. There are even some with a focus on privacy, like Proton Mail. But Gmail and similar services are overwhelmingly dominant, and not just because people are dumb. There is real value in having email that is accessible on any device, for free, with enough storage that you never really need to think about it. Proton offers 1GB for free now, which is pretty solid but a far cry from what Google, Microsoft, or Yahoo (yep they’re still around) provide. I mean, Google offered 1GB almost 20 years ago.

      I am personally still in the process of de-googling my life, and the idea of updating every account I ever signed up for using my gmail address is daunting. I’ll probably never get 100% of the way there; for now I am satisfied enough moving my personal correspondence and important accounts onto different email. Eventually I will probably set up my own domain and get a premium Proton plan so I won’t be too tightly tied to any particular email provider. Then if Proton ever enshittifies, I can take my domain and go elsewhere.