• Elvith Ma'for@feddit.org
    link
    fedilink
    arrow-up
    4
    ·
    8 days ago

    Yes and no. There are no guardrails on its output, so it will happily emit sudo rm -rf /*. But the “harness” of the agents gets to check if this gets executed. The default is usually “ask every time”, but you can change that to “everything is allowed” or “every invocation of ls and grep is allowed”. Problem is, when users get tired to constantly proof read and check every single command (and it will constantly emit commands) and then just hit “allow everything for this session” or such. Or when a user defines ls .* \| grep .* is a regex that defines that this is always OK to run but don’t see that it also matches any command after the grep that could do whatever with that data.

    Basically: the only safe way is to manually check every single command and you will be annoyed quickly and let you guard down.

    • flying_sheep@lemmy.ml
      link
      fedilink
      arrow-up
      3
      ·
      8 days ago

      Claude’s harness parses pipes and asks for permission for each individual command.

      Of course you’re right about the issues, e.g. when it creates a script file to execute and you allow that for the session, it can edit it and then re-execute it without your further input.

    • Mr_WorldlyWiseman@lemmy.blahaj.zone
      link
      fedilink
      arrow-up
      2
      ·
      8 days ago

      But the harness doesn’t actually prevent Claude Code from running commands. It just tells the model to end with a question instead of running the command directly. I haven’t actually read the source code though, just an anecdote from it running commands even when “ask every time” is enabled.

      • Elvith Ma'for@feddit.org
        link
        fedilink
        arrow-up
        1
        ·
        8 days ago

        Huh? I have access to Claude Code at work (and I’m not forced to use it for everything but rather as I see fit, yay!). It constantly asks me. BUT we get a custom config rolled out with it that is already pre populated with several commands and permissions for allow, deny and ask. Maybe it is a bit more lenient in the default mode?

        On the other hand, in the last thread of someone loosing their production server/database to Claude it was revealed that their cloud provider had some unintuitive settings that also immediately deleted all corresponding backups if you delete the resource - so I can see how this could have happened even when issuing commands manually.