Droids appear in practically every movie or TV series set in the \u201cStar Wars\u201d universe. They usually behave strangely. On the one hand, they give the impression of being independent-thinking beings with their own personalities; on the other, they\u2019re objects: they belong to someone, remain loyal to their owners, and carry out their orders. Most of the time we\u2019re never given any explanation for the droids\u2019 motivations. Why are some of them willing to break the law at their master\u2019s command? What determines who exactly they consider their master? How do they decide whom to remain loyal to and whose orders to follow?<\/p>\n
Someone might say, \u201cWhat\u2019s the difference?\u201d And from the perspective of the average viewer, they\u2019d be absolutely right. But from our perspective, the question of a droid\u2019s loyalty is first and foremost a question of cybersecurity. A droid is a complex cyber-physical system; by influencing its motivation, an attacker can gain access to confidential data, or even cause harm to the actual owner. In 2025, two TV series were released whose creators dealt with the issue of droid ownership. We were presented with two concepts for managing droid motivation. We\u2019ll attempt to examine both of these concepts and their shortcomings in this post. As usual, please be warned that the text may contain spoilers.<\/p>\n
In \u201cSkeleton Crew\u201d, we\u2019re introduced for the first time to the concept changing droids\u2019 behavior using voice commands. In several instances, a person who\u2019s not the droid\u2019s formal owner attempts to influence its actions by trying to mislead the droid. Overall, it appears this concept was influenced by modern chatbots based on large language models (LLMs) \u2014 it bears a striking resemblance to \u201cjailbreak\u201d attempts, i.e., attacks on the model aimed at bypassing security restrictions or built-in filters.<\/p>\n
Fern, a ten-year-old girl, wants her mother to think that she came home early and was studying in her room. But there\u2019s a problem: the home droid knows that\u2019s not true. So Fern uses the \u201cRun memory override\u201d command, and feeds the droid false information in the rather absurd phrasing, \u201cI was home, you just didn\u2019t see me\u201d.<\/p>\n
The fact that this method works points to two problems. First, the droid accepts the memory override command from Fern, which means it either lacks account control or has improperly configured permissions. The formal owner of the droid is the mother (otherwise, manipulating the memory would make no sense), but nevertheless, it accepts a potentially dangerous command from Fern. Second, a home droid tasked with watching over a child obviously lacks a built in parental control<\/a> feature.<\/p>\n The SM-33 droid considers the captain of the ship \u201cOnyx Cinder\u201d to be its owner. That is, it remains loyal not to a specific person, but to a role. A pirate code is used to determine the legitimacy of the right to hold this role. Unfortunately, the entire code isn\u2019t explained to us, but several of its tenets are cited. First, according to the SM-33\u2019s programming, there can be no ship without a captain (if there is no captain, someone must take their place). Second, the person who defeats the captain legally becomes the new captain. Third, if a challenge is invoked, the droid cannot assist the active captain, but must wait for the outcome of a duel. And fourth, a person can be the captain of only one ship \u2014 if a person takes command of another vessel, they automatically lose their status as captain of the first.<\/p>\n The SM-33 changes hands three times, strictly following this code. First, Fern lies to him, claiming she killed the previous captain and took his place. Then Jod Na Nawood throws down a challenge and becomes captain when Fern surrenders. Then Jod takes command of a pirate frigate and loses the captain\u2019s seat of the Onyx Ash, but manages to reclaim his rights.<\/p>\n And here\u2019s where an interesting twist occurs. Fern introduces a concept from children\u2019s games \u2014unclaimsies (essentially a reset of claims) \u2014 and asserts her own claim to the captain\u2019s seat. She then immediately orders SM-33 to throw the pirates overboard. To many viewers, this moment seemed extremely unrealistic \u2014 why would a droid, whose motivation is defined by the pirate code, consider such a transfer of rights to be legitimate? However, if we assume that the droids are controlled by LLMs, then this plot twist is quite explainable.<\/p>\n The Pirate Code is the original system of ethical values embedded in the droid. The chatbot typically assesses the interlocutor\u2019s intent at the very beginning of the dialogue, using a complex (resource-intensive) model for this purpose. Subsequently, to conserve resources and ensure safety during the conversation, simpler models are employed. However, the more context (dialogue history) there is, the more complex and resource-intensive it becomes to assess intent. This is precisely the basis of the popular jailbreak technique, which works on at least some modern LLMs. That is, as a result of prolonged communication with Fern, SM-33 lost the ability to correctly assess new requests for compliance with its original ethical guidelines, and therefore it deemed the statement about nullifying rights to be justified.<\/p>\n In fact, there is another issue with SM-33\u2019s security that\u2019s not directly dependent on whom it considers its owner, but is nonetheless related. The old captain gave the order to forget everything related to the planet At Attin, and to dismantle anyone who begins to take an interest in this matter. Fern, with the admin<\/span> captain\u2019s privileges, runs her favorite memory override, and forces the droid to retrieve its memories of At Attin, after which SM-33 recalls both the planet and the order to attack the questioner.<\/p>\n And as a result, we realize that, in fact, it did not carry out the old captain\u2019s order; the information about At Attin remained in the droid\u2019s memory; it simply couldn\u2019t find it \u2014 that is, if it did delete it, it was only from the index of accessible memories. Perhaps this is some physical property of the droid\u2019s memory, or maybe this can be explained by the fact that SM-33 was programmed not by a professional, but by a pirate. After all, its design includes other suboptimal solutions, such as a power switch accessible to anyone standing nearby, exactly like C-3PO\u2019s. But what makes sense for a protocol droid isn\u2019t exactly suitable for a combat droid designed, among other things, for hand-to-hand combat\u2026<\/p>\n In the series \u201cAndor\u201d, the prequel to the film \u201cRogue One,\u201d we finally see how the main character, Cassian Andor, acquired the reprogrammed Imperial security droid K-2SO to become his partner. And most importantly, the process of how the rebels changed his motivation is shown.<\/p>\n As it turns out, in order for a combat droid loyal to the Empire to stop obeying its original programming, its \u201ccortex\u201d must be replaced \u2014 though the replacement cortex can trigger rejection. The specialist says, verbatim: \u201cYou\u2019ll hear a lot of nonsense about reprogramming, which makes it sound as though it\u2019s a problem that can be solved from a console, but frankly, that\u2019s nonsense. It\u2019s really all about impulse suppression, which is entirely an engineering and wiring issue.\u201d<\/p>\n In other words, the rebels replace a certain component, after which the droid becomes a being with new moral principles. At the same time, it retains its memory (K-2SO later recalls how it once participated in a parade on Coruscant).<\/p>\n \u00a0<\/p>\n So, what conclusions can we draw from all this? Well, first, it becomes clear that a droid controlled by an LLM is a clear security threat. It can easily be misled and made to act against its rightful owner. And second, the hardware and software platform used to create droids in \u201cStar Wars\u201d is far from ideal. If our colleagues had been responsible for creating the droids, they\u2019d have strived to develop a cyber-immune solution in which functionality would be impossible after a key component was replaced, as would malicious memory manipulation. In other words, it\u2019s a real shame that a long time ago, in a galaxy far, far away, there was no KasperskyOS<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":" How and why droids from a galaxy far, far away switch their allegiances. <\/p>\n","protected":false},"author":700,"featured_media":35850,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1999,3020,3021],"tags":[2895],"class_list":{"0":"post-35849","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-business","8":"category-enterprise","9":"category-smb","10":"tag-mtfbwy"},"hreflang":[{"hreflang":"en-za","url":"https:\/\/www.kaspersky.co.za\/blog\/star-wars-droid-motivation\/35849\/"},{"hreflang":"en-in","url":"https:\/\/www.kaspersky.co.in\/blog\/star-wars-droid-motivation\/30467\/"},{"hreflang":"en-ae","url":"https:\/\/me-en.kaspersky.com\/blog\/star-wars-droid-motivation\/25513\/"},{"hreflang":"en-gb","url":"https:\/\/www.kaspersky.co.uk\/blog\/star-wars-droid-motivation\/30311\/"},{"hreflang":"ru","url":"https:\/\/www.kaspersky.ru\/blog\/star-wars-droid-motivation\/41790\/"},{"hreflang":"x-default","url":"https:\/\/www.kaspersky.com\/blog\/star-wars-droid-motivation\/55685\/"},{"hreflang":"ru-kz","url":"https:\/\/blog.kaspersky.kz\/star-wars-droid-motivation\/30621\/"},{"hreflang":"en-au","url":"https:\/\/www.kaspersky.com.au\/blog\/star-wars-droid-motivation\/36198\/"}],"acf":[],"banners":"","maintag":{"url":"https:\/\/www.kaspersky.co.za\/blog\/tag\/mtfbwy\/","name":"MTFBWY"},"_links":{"self":[{"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/posts\/35849","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/users\/700"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/comments?post=35849"}],"version-history":[{"count":0,"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/posts\/35849\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/media\/35850"}],"wp:attachment":[{"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/media?parent=35849"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/categories?post=35849"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kaspersky.co.za\/blog\/wp-json\/wp\/v2\/tags?post=35849"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Pirate droid SM-33: motivation<\/h3>\n
SM-33: Access to Memory<\/h3>\n
Season 2 of the series \u201cAndor\u201d<\/h2>\n