5G Vulnerability Analysis with Reinforcement Learning Toolbox - MATLAB & Simulink
Video Player is loading.
Current Time 0:00
Duration 19:35
Loaded: 0.84%
Stream Type LIVE
Remaining Time 19:35
 
1x
  • Chapters
  • descriptions off, selected
  • en (Main), selected
    Video length is 19:35

    5G Vulnerability Analysis with Reinforcement Learning Toolbox

    Ambrose Kam, Lockheed Martin Rotary and Mission Systems

    Published: 25 May 2022

    [AUDIO LOGO]

    Hi. My name is Ambrose Kam. I'm a Technical Fellow from Lockheed Martin. In my session here today, I would like to talk about 5G vulnerability analysis with MATLAB's Reinforcement Learning Toolbox. 5G is a transformational technology, and it's going to impact the way that we work, the way that we play, and the way that we fight as well.

    And so for this reason, we need to better understand the 5G implications and also what are some of the vulnerabilities associated with 5G. And more importantly, how can we better protect 5G as an infrastructure?

    So 5G as a whole has different layers, and 5G itself might include the end user equipment, the UE; the RAN, the Radio Access Network; the MEC, the Multi-Access Edge Computing; as well as the 5G core itself. And these are the major components of a typical 5G infrastructure.

    And so in order to better protect this 5G infrastructure, we need to understand the security implications. From our standpoint, because there are vulnerabilities, there are attack vectors. And so in order to understand these security concerns, we need to better understand what might be some of the vulnerabilities out there associated with 5G.

    And so we went through a lot of literature, including a lot of the standards bodies out there. This is one that I pull out from-- this is from the International Telecom Union, or ITU. And specifically, they have highlighted eight security dimensions which includes access control authentication, non-repudiation, data confidentiality, communication security, data integrity, availability, and privacy.

    Those are the major concerns as highlighted by the ITU organization. And it's really important to understand that because we all need to have a secure and persistent infrastructure. 5G is really not about-- it's not just about the fast download speed or the low latency time, but also, it's about security. So this is the reason why we need to investigate and understand some of the vulnerabilities associated with 5G.

    And so from that standpoint, we need to understand that security needs to be built into the system. It's not something that you would like to be bolted on later on. And also, it needs to be flexible because you have a lot of different deployment use cases. Also has to take advantage of some of the-- of these automated security features. For example, using AI machine learning, for example.

    And all of these security concerns need to be addressed early on in the lifecycle instead of trying to add security features after the 5G has been deployed.

    And so for this reason, we are using our synthetic simulation environment to better represent 5G infrastructure, and by using that, we can model some of those threat vectors that we were talking about, and some of the vulnerabilities as well. Here is another security framework.

    So this one came from the 3GPP. 3GPP is an organization for 5G. In this particular case, this technical spec came from R15, and the document itself included this diagram at the lower left-hand corner there that highlights some of the-- I guess the six security domains, which include network access, network domain, user domain, application domain, service-based architecture domain, visibility, and configurability.

    And these Roman numerals are in the diagram on the lower left-hand corner there. So as a reference, you can better understand some of the security concerns as pointed out by the organization. And as I said before, what we would like to do is to focus on the vulnerabilities associated with 5G in our synthetic environment, and also trying to model some of those threat vectors as well.

    Certainly, we can only model some of the known threat vectors at this point, but what we would like to do is to potentially use this digital twin environment that we have to expose the vulnerabilities so that we can understand some of the emerging threat vectors or attack vectors as well.

    This table I pull out from the IEEE paper, and it highlights some of the general 5G threat vectors. It includes threat vectors that are focusing more at the physical layer, the RF layer, Radio Frequency layer, as well as some of the infrastructure like network layer or IP layer, for example.

    And so you see, for the physical layer attacks, it might include jamming, eavesdropping, scanning attacks, man-in-the-middle attack, EMC attack, and so on. For network and IP layer attacks, there is the unauthorized access configuration attacks, key exposure, TCP-level attacks, so on and so forth.

    The idea is that we'd like to better understand them so that we can represent them in our network digital twin environment that we set up. There are many-- again, there are many ways to model these attack vectors. You can use a 5G cyber range capability to model these attack vectors. You can also do what we do, which is focusing more on the model and simulation aspect of it.

    There's no right or wrong. We chose this 5G modeling simulation because we are familiar with some of the tools here, and we can also leverage our AI machine learning capabilities that we already have our expertise.

    In our environment, we are using this commercial off-the-shelf tool called Exata. And in this case here, we are using Exata to model those key 5G components to UE, which is the User Equipment, the RAN, Radio Access Network, the MEC, the Multi-Access Edge Computing, as well as the 5G core.

    And so what we'd like to do is to set up both the 5G infrastructure itself, and also trying to model some of the traffic that might be going on through that infrastructure. And so once we have the digital representation of the infrastructure and the traffic model, now we can look at modeling some of the vulnerabilities that we have pointed out.

    The tool itself allows you to model vulnerabilities at different levels. One is at the host node level. So here, this is a snapshot of the tool. It provides you a GUI environment that you can input these vulnerabilities. And also, it also provides that user behavioral model as well as well as the operating system resource model, which includes the memory, this CPU, and your application, and so on.

    So this is how you would specify the 5G infrastructure itself, as well as the individual nodes. And then we can also use the same tool to model the cyber attacks. And in this case here, there are different ways to model cyber attacks in this case here. We are focusing more on the hardware and the-- or HITL. And we focus on a select few of these attack vectors.

    The idea here isn't so much to boil the ocean, but we wanted to have some kind of representative attack vector so that we can explore the use of this reinforcement learning and see if we could optimize the attack vectors, but also trying to optimize the mitigation schemes as well.

    The other way that we can also model is through the use of the attack template, and also by modeling the exploit's vulnerability explicitly. For our reinforcement learning, we have selected the MATLAB Reinforcement Learning Toolbox. And in this case here, this Reinforcement Learning Toolbox-- so inside the toolbox, we have created a learning agent. And then the learning agent will make a selection of the attack vectors, and these attack vectors will be fed into the Exata environment.

    And like I said earlier, we are using this human-in-the-loop interface to provide-- to input those attack vectors so that Exata model can execute and provide the feedback through the observations. And the Reinforcement Learning Toolbox will calculate based on the observations of the attack vectors, whether or not they succeed or not. It will calculate the reward points, and that feedback loop goes over again.

    And so that's basically the process. Here is another diagram that gives you a little bit more visibility as far as what's going on inside that MATLAB reinforcement learning agent. As I mentioned earlier, the learning agent is composed of the policy and also the reinforcement learning algorithm. And so it interacts with the external model-- in this case, Exata-- through the use of the human-in-the-loop configuration file.

    Basically, we are using the learning agent to select these attack vectors, and these attack vectors will be captured in this configuration file, and then Exata is going to read the configuration file, and then it's going to produce the output through the use of the Exata stat file.

    And then the learning agent will pass through the STAT file understand what has happened what worked, what didn't work in that previous episode, and then calculate the reward points. And then the learning agent is going to go through the feedback loop again for the next set of actions in the subsequent episode.

    So over time, the learning agent is going to learn, and hopefully by looking at the reward points that they got from the previous episodes, it's going to improve the selection of these attack vectors. And ultimately, it's going to come up with the best solution, the most optimum solution.

    So here's one example of the results, of the learning results through the use of the Matlab Reinforcement Learning Toolbox. The MATLAB Reinforcement Learning Toolbox gives you the capability to explore different techniques. And in this case here, we are using Deep Q-Network. And you can also use other algorithms as well, but we noticed that by using the DQN, it provides the best results and it converges quicker than some of the other approaches.

    And so the idea is that we would like to use this training results which highlights the number of reward points that the learning agent is accumulating over time. And then you can see that the reward points is trending upward. It's because it's learning what are the effective attack vectors, and then over time, it's going to build up.

    So this is, from the visual standpoint, you as the end user, you can actually see how the learning is actually going. And that's actually one of the major advantages of MATLAB Reinforcement Learning Toolbox, is because it's got a lot of these built-in features and capabilities. A lot of the capabilities you can just call directly from within the model so you don't have to write your own code to develop this.

    So that concludes our presentation, but I just wanted to highlight some of the future work that we have in mind. This particular example that I shared with you just now is focusing more on the single agent. What we would like to do is to explore this to include multiple agents-- not just one agent, but more than one agent so that we can provide a little bit more realism.

    And that can be done through the use of Simulink. That's another product from MATLAB. And through the use of these Multi-Agent Reinforcement Learning Framework, we can specifically use a specific agent to model a category of attack vectors, for example, or maybe a category of defensive mitigation schemes. And that's really the power of the Multi-Agent Reinforcement Learning Framework.

    Secondly, what we would like to do is to leverage industry standards. For example, there is the MITRE's ATT&CK Framework, which is focusing more on the attack-- the offensive tactics and techniques. Subsequently, a year ago, MITRE has also released what's called a D3FEND Framework, which highlights the defensive tactics and techniques.

    And so the cyber industry has been using both of these frameworks, and what we would like to do is to include these two frameworks in our learning model.

    So it's not just the cyber attack model from within Exata, but also wanted to branch out to other offensive techniques or defensive techniques so that we can cast a wider net and be able to have the flexibility that we need to model different attack vectors and also different mitigation schemes, for example, so that we can look at the effectiveness of these mitigation schemes, essentially.

    Related to that is that we would like to use other tools, other simulation tools, and introduce them to our synthetic simulation framework that I shared with you earlier. So we can introduce Cyber Attack Network Simulation, which is Lockheed Martin's in-house-built capability.

    We can also introduce AFSIM, which is Advanced Framework for Simulation Integration and Modeling. It's Air Force Research Lab simulation tool. We would like to introduce CANS because we would like to better understand-- or better model some of the network-level cyber attack vectors, for example.

    And-- so and then for AFSIM, we would use it in terms of using as a platform for mission-level analysis. A lot of our stakeholders, they want to better understand how the 5G threat vectors could impact their particular missions. They may not care about specific component being compromised. They do care about what is the performance outcome, how does it impact the overall mission. So we would like to be able to have that kind of flexibility as well.

    And so we-- so in the past, we have actually worked with the MathWorks team to introduce CANS, and also AFSIM, in our simulation framework. What we haven't done is to actually run all three tools together, the Exata, CANS, and AFSIM together in this simulation environment, which is driven by the Reinforcement Learning Agent. And so we would like to explore that.

    Related to that is that we would like to have better modeling capability with respect to the physical layer of 5G. That includes the waveform, the beamforming aspect, and how to optimize the physical transmission, for example. And so we would like to employ the 5G Toolbox as well in the near-future.

    And additionally, along with what I just said, we would like to be able to combine the cyber effects as well as some of the electronic warfare effects like jamming and so on, eavesdropping, and maybe some kind of RF-level-- radio frequency-level interference or something like that. So that-- with that introduction-- with the introduction of the 5G Toolbox, that can help us model some of these advanced capabilities.

    So with that, this is my last shot. And I would like to thank MathWorks for giving me this opportunity to discuss this 5G vulnerability analysis capability with you. And we would like to continue with this work and continue to work with the MATLAB Technical Support Team as well for some of the future work that I discussed earlier. So thank you.