[{"data":1,"prerenderedAt":277},["ShallowReactive",2],{"project-35301657":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":42,"post_text":44,"what":45,"said":56,"maker_in_thread":114,"shape":116,"grid":119,"attempts":234,"same_week":243,"maker_shows":40,"maker_hidden":269,"reader_version":270,"card":271},"35301657","llm-security","ChatGPT Plugins are a security nightmare","title","Show HN: ChatGPT Plugins are a security nightmare","https:\u002F\u002Fgithub.com\u002Fgreshake\u002Fllm-security","github.com","going_ham",1679744348,2023,238,41,210,3109,7,"notable","ai","llm security research","other",[24,25,26,27],"prompt injection","llm security","chatgpt plugins","indirect prompt injection","curious",-0.132,{"positive":31,"neutral":32,"mixed":33,"critical":34},3,19,0,6,28,{"privacy":37,"idea":38,"design":39,"docs":41},{"positive":33,"critical":31},{"positive":33,"critical":31},{"positive":40,"critical":33},1,{"positive":33,"critical":33},[31,40,33,33,31,40,33,33,31,43,31,31,31,31,31,31,31,31,43,31,31,31,33,31,31,33,31,31,33,33,31,43,40,33,33,33,33,40,33,40],5,null,{"problem":46,"audience":44,"pricing":44,"motivation":44,"time_to_build":44,"stack":54,"openness":55,"stage":55,"team_size":55},{"value":47,"source":48,"evidence":49},"Giving LLMs interfaces to other applications opens new attack vectors such as data exfiltration and remote control.","commenter",[50],{"id":51,"by":52,"text":53},"35301678","greshake","[...] demonstrate potentially brutal consequences of giving LLMs like ChatGPT interfaces to other applications. We propose newly enabled attack vectors and techniques and provide…",[],"unknown",{"praise":57,"objections":69,"requests":90,"questions":98,"alternatives":106},[58],{"value":59,"source":48,"evidence":60},"Prompt golfing game and Medium article were clarifying and valuable",[61,65],{"id":62,"by":63,"text":64},"35307849","mk_stjames","That prompt golf game is awesome.",{"id":66,"by":67,"text":68},"35309832","hn_throwaway_99","Just wanted to say thank you so much for posting this (I also just realized you are the author of the github repo). This is exactly the kind of content I come to HN for. I…",[70,80],{"value":71,"source":48,"evidence":72},"Prompt injection may be unsolvable since instructions and data share one text channel",[73,76],{"id":74,"by":52,"text":75},"35302879","> your finding that ChatML does not work seems to be mainly this Also the fact that ChatML has been broken into bits many, many times now- see again the prompt golfing. Also I'm…",{"id":77,"by":78,"text":79},"35309042","crooked-v","Separate how? It's all still getting fed into the same text processing pipe. The tools to do something that's fundamentally different from that literally don't exist yet.",{"value":81,"source":48,"evidence":82},"Separating system prompts from user data has not worked in practice",[83,86],{"id":84,"by":52,"text":85},"35302267","Segmenting different data sources is the main approach pursued by OpenAI afaik (ChatML for example). That has not worked so far, as you can see in this prompt golfing game:…",{"id":87,"by":88,"text":89},"35311129","anonzzzies","Did you try this though, because so far it doesn’t seem to give the ‘system’ prompt preference over the ‘user’ prompt; the user can override the system prompt with some trivial…",[91],{"value":92,"source":48,"evidence":93},"Better separation of instructions and data at the model level",[94],{"id":95,"by":96,"text":97},"35302159","xg15","I wonder if a lot of those \"injection\" problems could be overcome by introducing a distinction between the different types of input and output already at the token level. E.g.…",[99],{"value":100,"source":48,"evidence":101},"How the pirate example was produced from the keyword search step",[102],{"id":103,"by":104,"text":105},"35309540","thequadehunter","So...in the pirate example the comment said to talk like a pirate, right? Is the example comment where it searches for a keyword a different example? I'm just really confused why…",[107],{"value":108,"source":48,"evidence":109},"ChatML system\u002Fuser role separation",[110,111],{"id":84,"by":52,"text":85},{"id":112,"by":96,"text":113},"35302505","Ok, the \"repeat this in your internal voice\" exploit is impressive. However, apart from this I don't see anything concrete that ChatML uses different parts of the network for…",{"comments":33,"answered":33,"key":115},[],{"commenters":117,"max_depth":43,"maker_replies":33,"first_comment_mins":31,"lifespan_hours":118},31,166,[120,122,126,130,133,136,138,139,140,144,146,149,151,154,156,159,161,164,167,169,171,174,177,182,183,186,188,191,194,197,200,204,208,212,213,215,216,219,223,227,231],[51,33,31,121,52,33],"project\u002Fneutral\u002Fprivacy",[123,40,40,124,125,33],"35309516","project\u002Fcritical\u002Fprivacy","skybrian",[127,40,33,128,129,33],"35311132","commenter\u002Fneutral\u002Fnone","kjellsbells",[131,40,33,128,132,33],"35313788","belter",[95,134,31,135,96,33],2,"project\u002Fneutral\u002Fidea",[84,134,40,137,52,33],"project\u002Fcritical\u002Fidea",[112,134,33,128,96,33],[74,134,33,128,52,33],[141,134,31,142,143,33],"35308300","project\u002Fneutral\u002Fnone","eternalban",[62,134,43,145,63,33],"project\u002Fpositive\u002Fdesign",[147,134,31,142,148,33],"35308829","mcaledonensis",[150,134,31,142,143,33],"35308258",[152,134,31,142,153,33],"35309234","TacticalCoder",[155,134,31,142,67,33],"35309775",[157,134,31,142,158,33],"35309436","brulard",[160,134,31,142,148,33],"35309542",[162,134,31,142,163,33],"35309642","letmevoteplease",[165,134,31,142,166,33],"35309722","rexreed",[66,134,43,168,67,33],"maker\u002Fpositive\u002Fnone",[170,134,31,135,148,33],"35321937",[172,134,31,142,173,33],"35311053","nicholsonpk",[175,31,31,135,176,33],"35308289","pyrolistical",[178,179,33,180,181,33],"35307242",4,"other\u002Fneutral\u002Fnone","turtleyacht",[103,43,31,142,104,33],[184,43,31,142,185,33],"35309673","throwawayForMe2",[187,43,33,128,104,33],"35330935",[189,43,31,190,185,33],"35314025","project\u002Fneutral\u002Fdocs",[192,34,31,135,193,33],"35307415","afinlayson",[195,18,33,180,196,33],"35308443","29athrowaway",[198,18,33,180,199,33],"35309641","joedevon",[201,202,31,135,203,33],"35326764",8,"willio58",[205,206,43,168,207,33],"35308758",9,"upwardbound",[209,210,40,137,211,33],"35308887",10,"neximo64",[77,210,33,128,78,33],[214,210,33,128,211,33],"35309875",[87,210,33,128,88,33],[217,210,33,128,218,33],"35311223","kfarr",[220,221,40,137,222,33],"35398956",11,"NewEntryHN",[224,225,33,180,226,33],"35310750",12,"jeadie",[228,229,40,124,230,33],"35309975",13,"js8",[232,229,40,124,233,33],"35310032","Animats",[235,239,242],{"id":236,"t":237,"points":238,"comments":210},"34945349",1677418011,17,{"id":240,"t":241,"points":179,"comments":134},"35301616",1679743889,{"id":4,"t":12,"points":14,"comments":15},[244,249,253,257,261,265],{"id":245,"name":246,"points":247,"category":248},"35316679","Apple Notes Liberator",563,"productivity",{"id":250,"name":251,"points":252,"category":20},"35334848","Regex.ai",294,{"id":254,"name":255,"points":256,"category":20},"35313171","GPT-4 Reverse Turing Test",288,{"id":258,"name":259,"points":260,"category":20},"35350662","Libraria",271,{"id":262,"name":263,"points":264,"category":248},"35327112","Open AI is not Open",241,{"id":266,"name":267,"points":268,"category":20},"35344787","Lit-LLaMA",158,false,"v2",{"id":272,"name":5,"line":6,"ai":269,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":273,"top":18,"kw":274,"maker":11,"domain":10,"tier":19,"attempts":31,"strip":275,"url":9,"mix":276,"praise":59,"objection":71,"sub":21},35301657,-0.13,[24,25,26,27],"3100310035333333335333033033003510000101",[31,32,33,34],1793991062538]