Just a regular Joe.

  • 0 Posts
  • 13 Comments
Joined 3 years ago
cake
Cake day: July 7th, 2023

help-circle

  • To be fair, protecting credentials and important data is the company and individual’s responsibility. The building blocks to restrict access are there, but are often not leveraged (even by large companies with the ability to invest)

    Sandboxing is one of them: Both Codex & Claude’s sandboxing is reasonable (sandbox-exec, Linux cgroups & seccomp). Many others are lacking, sometimes deliberately.

    I do most coding with Pi these days, and I have it heavily sandboxed. I expose sensitive services via a localhost network service with auth (typically for running scripts outside the sandbox). Reads are limited to the system binaries/libs, nad writes to the project dir & Pi’s own dirs. If I choose to give a particular session creds, then I have to be very deliberate. I also force egress traffic through a proxy (just logging for now, but I have plans)



  • $14000 in API pricing is not $14000 in costs, though. Costs are hard to calculate because of the huge capital outlays and unknowns about hardware lifecycles, various business deals, and limited public knowledge.

    It’s likely that inference costs for good-enough models will go down over time. China’s API pricing tells us the direction already. Energy costs will be a driving factor in the west, I guess.

    So… they are almost certainly subsidizing plans right now, but on average, it won’t be by sooo much. Your average ChatGPT user will hardly use Codex, for example. Your average developer is not token-maxxing either.

    Why are they subsidizing plans? To build a sticky customer base … which means they want you to stick to their tools - their coding agents/harnesses, their integrations, etc. Models are/will be increasingly interchangeable, so they are building sticky ecosystems instead.


  • A couple of tricks I use:

    • an apparmor profile tied to a shell script that wraps other commands … it restricts read & write access to a scratch directory … perfect for builds or one off scripts.

    • iptables rules & cgroups to restrict network access… I have a setuid wrapper that drops privs again…

    • bwrap and mounting only what’s necessary… quick to get going.

    • custom landlock wrapper, similar to apparmor but allows for quick userspace wrapping.

    They can be combined too.






  • In many cases, yes. A difference now will be the long-term size and composition of the teams (smaller & more generalists, with PMs, POs & Architects just as likely to contribute code as engineers)

    2 pizza teams can become 1 pizza teams who can manage an entire product/component, or more. And those 3+ pizza teams can strip the fat or split into more productive teams.

    I think we’ll also see increased demand for platform/deployment standardization and concentrated/novel support structures, as teams start biting off more than they can chew, along the the desire for out-of-the-box guardrails around AI code & tools.



  • Given that AI is particularly useful at increasing alignment (when applied smartly), and that this is often a role delegated to middle managers, it is quite likely that flatter orgs will happen.

    The need for top-tier technical, product, and business judgement and problem engagement will increase, while the need for muddle-through managers and similar roles will decrease.

    We’ll see more initiatives organized end-to-end by small groups of smart people, with virtual teams/coalitions forming to bypass “archaic” processes and deliver meaningful results. We’ll see a lot of sloppy failures along the way too, but the overall trend seems clear.