Operation: Re-Boot
When a leader suddenly disappears, things go south. This is precisely what happened in my Arma 3 unit. Things went south.
Our unit leader vanished from one day to the next, leaving us all in the dark. Nobody really knew what was going on. Not good. Let me tell you why.
Much of our unit's operations were manual processes. Ewww, right?!
One significant area impacted by our leader's departure was the recruiting process. This process typically unfolded as follows:
- A new candidate joins our Discord server.
- The candidate chats with the unit leader, and they schedule a meeting.
- During the meeting, they go through various tasks and questions about Arma 3.
While this is a simplified explanation, the onboarding process usually took about 30 Minutes. Now, nobody is here to fill in this void. We need a solution. As always better yesterday, than today.
A Spark of Innovation
Well, it is not all bad. Since there is nobody at the top holding the creative people back (e.g. me), some stuff can actually change for the better.
One of these things is the onboarding process. I'm taking care of this!
What do you think I'm going to do now? Sit down and go through the training with strangers on the internet?? Nope, fuck that. I'm going to automate that shit.
Automated Bootcamp
No no, I don't think you understand. I'm not just building a mission that every newcomer has to go through and then it is all good and toasty. As a developer, I don't need extra human interaction, especially if I can automate processes. I'm talking about the real deal.
What does this mean? Let me explain by showing you first the current diagram:
So, wtf is going on here? The flow isn't that complicated at all. The magic part is actually Pythia.
This mod allows you to execute Python scripts from Arma. Pretty dope, right?! At that point, you can do pretty much anything you like.
In my case, I use it as an API to a Flask app, which is also an API to my database. Because you can't have enough APIs in your Architecture
As for the database, let's keep it simple: Pocketbase!
PocketBase is an open source backend consisting of embedded database (SQLite) with realtime subscriptions, built-in auth management, convenient dashboard UI and simple REST-ish API.
Wait a minute
Some of you might ask now: Why not simply go the direct route from Pythia to the database or simply use INIDBI2?
INIDIBI2 is out right away. We are running on a Linux server baby. INIDIBI2 has only support for Windows.
Ok, but about the direct route from Pythia to the database? I've had two reasons. First, this way is easier to debug any issues and second, we want later to enhance the Flask app to do more, e.g. connect with Discord.
The flow
The player plays the mission on our server. Certain events trigger Pythia to send data over to Flask, which is then stored in the database. Simple, yet effective.
Code in Boots
Disclaimer:
Before I show push you in the code I want to make clear, at this point the project is a Beta version. We are at the stage of Make it work. Also, I'm not coding that much in SQF. Expect garbage code, a lot of garbage code.
This out of the way, let me show you the code:
/initPlayerLocal.sqf
if !(isMultiplayer) then
{
["Play this in Multiplayer", true, 3] remoteExec ["BIS_fnc_endMission", 0, true];
};
waitUntil {
!isNull findDisplay 46
};
RCT7playerData = nil;
[] execVM "scarCODE\ServerInfoMenu\sqf\initLocal.sqf";
[player] remoteExec ["RCT7_getFromDb", 2];
[] spawn {
waitUntil {
sleep 3;
{
publicVariable "RCT7playerData"
} remoteExec ["call", 2];
!(isNil "RCT7playerData");
};
createDialog 'RscDisplayServerInfoMenu';
private _prefix = "Start";
if (count RCT7playerData > 0) then {
_prefix = "Resume";
};
_actionTitle = [_prefix, "Bootcamp"] joinString " ";
player addAction [_actionTitle,
{
params ["_target", "_caller", "_actionId", "_arguments"];
player removeAction _actionId;
[] spawn RCT7Bootcamp_fnc_init;
}
];
};
Server-side
The most interesting part above is [player] remoteExec ["RCT7_getFromDb", 2]; so let me show you the code behind it:
/initServer.sqf
RCT7_dbQueue = [];
RCT7_writeToDb = {
params["_player", "_section", "_key", "_value", ["_additional_pairs", []]];
if !(isServer) exitWith {};
_section_data = [ [_key, _value] ]; // Create section data array with the given key-value pair
// Add additional key-value pairs if any
{
if (typeName _x == "ARRAY" && count _x == 2) then {
// Check if it's a valid key-value pair
_additional_key = _x select 0;
_additional_value = _x select 1;
_section_data pushBack [_additional_key, _additional_value];
};
} forEach _additional_pairs;
_data = [_section, _section_data];
["bootcamp.add", [getPlayerUID _player, name _player, _data]] call py3_fnc_callExtension;
};
RCT7_getFromDb = {
_player = param[0, objNull, [objNull]];
if !(isServer) exitWith {};
_data = ["bootcamp.get_data", [getPlayerUID _player]] call py3_fnc_callExtension;
RCT7playerData = _data;
// no data could be found, so we assume a new recruite
// Potential problem: DB and/or Flask are not running
if (_data # 0 # 0 isEqualTo "error") exitWith {
RCT7playerData = [];
};
publicVariable "RCT7playerData";
};
RCT7_addToDBQueue = {
_item = param[0, [], [[]]];
RCT7_dbQueue pushBack _item;
};
_runQueue = {
[] spawn {
while { true } do {
sleep 1;
if (count RCT7_dbQueue isEqualTo 0) then {
continue;
};
_currentItem = RCT7_dbQueue # 0;
_currentItem call RCT7_writeToDb;
RCT7_dbQueue deleteAt 0;
};
};
};
call _runQueue;
So the server side on Arma is simply responsible for initializing the player with the proper data (stored in RCT7playerData). During the mission, the server then simply receives data from the client and pushes it to the flask API to be stored in the database.
Because I also ran into race conditions when it comes to writing to the database I've added a queue, so every new entry is added correctly in the database.
Don't judge me. Yes, I know it should be handled by the Flask app, but it was easier to do it here, so I'll take this as a technical debt and move this when I'm going to refactor the code.
Pythia-side
As Pythia dictates the setup I've created an empty mod with my Python code.
Behold my mighty Python code:
/bootcamp.py
import json
import threading
import http.client
import socket
import encodings.idna
URL="127.0.0.1"
PORT=5001
def send_post_request(playerUID: str, playerName: str, data: dict):
endpoint = "/bootcamp"
data = {"playerUID": playerUID, "playerName": playerName, "data": data}
body = json.dumps(data).encode("utf-8")
headers = {"Content-Type": "application/json", "Content-Length": len(body)}
try:
conn = http.client.HTTPConnection(URL, PORT, timeout=0.1)
conn.request("POST", endpoint, body, headers)
conn.getresponse() # We're not waiting for the response
except (http.client.HTTPException, socket.timeout):
pass
finally:
conn.close()
def process_nested_data(data):
if isinstance(data, dict):
return [[key, process_nested_data(value)] for key, value in data.items()]
elif isinstance(data, list):
return [process_nested_data(item) for item in data]
else:
return data
def get_data(playerUID: str):
endpoint = f"/bootcamp/{playerUID}"
try:
conn = http.client.HTTPConnection(URL, PORT)
conn.request("GET", endpoint)
response = conn.getresponse()
if response.status != 200:
return [["error", f"Could not get data for playerUID {playerUID}"]]
json_data = json.loads(response.read().decode("utf-8"))
return [[key, process_nested_data(value)] for key, value in json_data.items()]
except (http.client.HTTPException, socket.timeout):
return [["error", f"Could not get data for playerUID {playerUID}"]]
finally:
conn.close()
def convert_list(lst):
if len(lst) < 2:
raise ValueError("Input list has an incorrect format")
section_name = lst[0]
key_value_pairs = lst[1]
res_dct = {section_name: {}}
for pair in key_value_pairs:
if len(pair) == 2:
key, value = pair
res_dct[section_name][key] = value
else:
raise ValueError("Each key-value pair must have exactly 2 elements")
return res_dct
def add(playerUID: str, playerName: str, data: list):
post_thread = threading.Thread(
target=send_post_request, args=(playerUID, playerName, convert_list(data))
)
post_thread.start()
return True
```The only functions used in Arma are get_data to get the user data and add to add something to the database.
I know. I know. It is not good, but it works, so it is good.
No requests? Yes!
My goal here was to keep the Python environment as clean as possible. Reason: Performance. While I was testing my scripts requests the initial loading of the data was a lot slower than without it, so I decided to ditch it.
Because I also use asynchronous requests here I don't need to worry about waiting for a response from the Flask API. The only downside of this is, the DB or Flask might crash and the process won't be stored.
How do I plan to fix this? Like in the old saying: Don't fix it, when it's not broken. I'm not going to solve this until it actually appears.
Flask-side
The Flask app is as simple as it can be. I'm just showing it, so you have the full picture:
/app.py
from flask import Flask, request, jsonify
from pb import client
collection = client.collection("bootcamp")
app = Flask(__name__)
@app.route("/bootcamp", methods=["POST"])
def index():
data = request.get_json()
print(data)
uid = data["playerUID"]
record = collection.get_list(1, 1, {"filter": f"playerUID = {uid}"})
if not record.items:
collection.create(data)
return {"message": "OK"}
item = record.items[0]
if item.data is None:
item.data = {}
section = list(data["data"].keys())[0]
if section not in item.data:
item.data[section] = {}
for key, value in data["data"][section].items():
if key in item.data[section] and isinstance(value, list):
item.data[section][key].extend(value)
else:
item.data[section][key] = value
updated_item_data = {
"playerUID": item.player_uid,
"playerName": item.player_name,
"data": item.data,
}
collection.update(item.id, updated_item_data)
return {"message": "OK"}
@app.route("/bootcamp/<playerUID>", methods=["GET"])
def get_player_data(playerUID):
record = collection.get_list(1, 1, {"filter": f"playerUID = {playerUID}"})
if not record.items:
return {"message": "Player not found"}, 404
item = record.items[0]
data = jsonify(
{
"playerUID": str(item.player_uid),
"playerName": item.player_name,
"data": item.data,
}
)
return data
if __name__ == "__main__":
app.run()
/pb.py
from pocketbase import PocketBase
import os
client = PocketBase("http://127.0.0.1:8090")
admin_data = client.admins.auth_with_password(
os.environ["PB_USERNAME"], os.environ["PB_PASSWORD"]
)
Easy right? Let's move on to the fun part now!
The BootCamp
The BootCamp consists of the following sections:
- Radio Familiarization
- Understanding 3D Reports
- Throwing Grenades
- Map Reading
- Close Quarters Combat
- Anti-Tank Usage
- Anti-Air Usage
- Formations
- Bounding
- ACE Medical
Let me show you the 3D Report section because it covers everything in one go:
/functions/trainings/fn_mapTrainingModule.sqf
/*
Author: Eduard Schwarzkopf
Description:
Map training module
Parameter(s):
0: Object - module that has all elements synched to it
Returns:
true
*/
// Initialize variables and objects
_module = param[0, objNull, [objNull]];
_syncedObjects = synchronizedObjects _module;
private _triggerObj = nil;
private _targetController = nil;
private _targetClusterList = [];
// Iterate through synced objects and classify them
{
private _syncedObj = _x;
// Check if object is a Target Cluster
if (_syncedObj isKindOf "Logic" && ["TargetCluster", str _syncedObj] call BIS_fnc_inString) then {
_targetClusterList pushBack _syncedObj;
continue;
};
// Check if object is a Target Controller
if (_syncedObj isKindOf "Logic" && ["TargetController", str _syncedObj] call BIS_fnc_inString) then {
_targetController = _syncedObj;
continue;
};
} forEach _syncedObjects;
// Initialize Target Controller
[_targetController, 0] call RCT7Bootcamp_fnc_handleTargets;
// Initialize fired event counters
firedCount = 0;
_firedIndex = player addEventHandler ["Fired", {
firedCount = firedCount + 1;
}];
// Initialize and iterate through target clusters
_index = 0;
_magSize = getNumber (configfile >> "CfgMagazines" >> (getArray (configFile >> "CfgWeapons" >> currentWeapon player >> "magazines") # 0) >> "count");
_count = count(_targetClusterList);
// Initialize earplug task
call RCT7Bootcamp_fnc_earplugTask;
// Create the main task for Map Training
private _mainTask = "MapTraining";
[_mainTask, "Finish the Map Training", "Follow the instructions", "intel", "CREATED", true, true, -1] call RCT7Bootcamp_fnc_taskCreate;
// Start the player section
player call RCT7Bootcamp_fnc_sectionStart;
// Iterate through target clusters and manage shooting tasks
while { _count isNotEqualTo _index } do {
1 call RCT7Bootcamp_fnc_handleMags;
_targetCluster = _targetClusterList select _index;
// get list of valid targets
_targetList = [];
{
if (_x isKindOf "TargetBase") then {
_targetList pushBack _x;
};
} forEach (synchronizedObjects _targetCluster);
// set up variables for shooting tasks
_targetCount = count(_targetList);
_invalidTargetCluster = + _targetClusterList; // copy array
_invalidTargetCluster deleteAt _index;
_grid = mapGridPosition (_targetList # 0);
// Initialize database section name
dbSectionName = ["Grid", _grid] joinString "-";
// Initialize shot counters
shotsValid = 0;
shotsInvalid = 0;
_shotsMissed = 0;
firedCount = 0;
// set up task description and subtask variables
_taskDescription = ["Shoot all", _targetCount, "targets at grid:<br/><br/>", _grid select [0, 3], _grid select [3, 5]] joinString " ";
_subTaskId = ["TargetCluster", _index] joinString "_";
_subTaskTitle = [_index + 1, "Hit the correct Targets"] joinString " - ";
// Create subtask
systemChat ([_subTaskId, _mainTask] joinString ".....");
[[_subTaskId, _mainTask], _subTaskTitle, _taskDescription, "search"] call RCT7Bootcamp_fnc_taskCreate;
// Add event handlers for invalid targets
{
{
_invalidTarget = _x;
_invalidTarget addMPEventHandler ["MPHit", {
params ["_unit", "_source", "_damage", "_instigator"];
// invalid
if (_unit animationPhase "terc" isEqualTo 0) then {
player call RCT7Bootcamp_fnc_targetHitInvalid;
shotsInvalid = shotsInvalid + 1;
_grid = mapGridPosition _unit;
[[player, dbSectionName, "shotsInvalid", shotsInvalid, [["wrongTargetList", [_grid]]]]] remoteExec ["RCT7_addToDBQueue", 2];
_unit removeAllMPEventHandlers "MPHit";
};
}];
} forEach synchronizedObjects _x;
} forEach _invalidTargetCluster;
// Add event handlers for valid targets
{
_x addMPEventHandler ["MPHit", {
params ["_unit", "_source", "_damage", "_instigator"];
player call RCT7Bootcamp_fnc_targetHitValid;
shotsValid = shotsValid + 1;
[[player, dbSectionName, "shotsValid", shotsValid]] remoteExec ["RCT7_addToDBQueue", 2];
_unit removeAllMPEventHandlers "MPHit";
}];
} forEach _targetList;
// Wait until all targets are hit or player runs out of ammo
_time = time;
waitUntil {
_targetCount isEqualTo shotsValid || _magSize isEqualTo firedCount
};
// set subtask state and update index
[_subTaskId] call RCT7Bootcamp_fnc_taskSetState;
_index = _index + 1;
_shotsMissed = firedCount - (shotsInvalid + shotsValid);
[[player, dbSectionName, "shotsMissed", _shotsMissed, [["time", time - _time - 2]]]] remoteExec ["RCT7_addToDBQueue", 2];
sleep 1;
// Reset target animations and remove event handlers
(synchronizedObjects _targetController) apply {
_x animate["terc", 0];
_x removeAllMPEventHandlers "MPHit";
};
// Show hint for the next target cluster
if (_count > _index) then {
["Next in", 3] call RCT7Bootcamp_fnc_cooldownHint;
};
};
// Remove player fired event handler
player removeEventHandler ["Fired", _firedIndex];
// set target controller state and finish the section
[_targetController, 1] call RCT7Bootcamp_fnc_handleTargets;
player call RCT7Bootcamp_fnc_sectionFinished;
[_mainTask, "SUCCEEDED", true, true] call RCT7Bootcamp_fnc_taskSetState;
0 call RCT7Bootcamp_fnc_handleMags;
true;
The flow is actually simple. Let me show you the setup:
A little bit of explanation. The module is just an object that has everything synched together. This way I'm able to get all the other objects in the code without using global variables all over the place.
The TargetController has all pop-up targets synched. The reason is so I can control the state of the targets in one place. This means I can pop them up and down how I need this, e.g. when the section is started or finished. Nothing really fancy here.
Now for the more interesting part: Each of the TargetCluster Modules are synched with popup targets. In this case 3 targets on each Cluster. Then the loop goes through each cluster of targets and tells the player to shoot those targets at their grid.
The player gets the task of finding the correct cluster of targets and hitting all of them. Each shot is registered if it is a valid hit, missed shot, an invalid hit and which invalid cluster was hit. While I'm on it I also track the time the player needed to complete each section.
All sections are basically the same and the code is not that complicated once you get the idea. You can boil it down to the following components:
- Creating tasks a player needs to follow
- Wait until the task is complete
- Send the data to the server so it can be stored in the database
- Give the next task until the end of the section is reached
Mission Accomplished: A New Era for the Unit
In conclusion, Operation: Re-Boot demonstrates how the sudden disappearance of a leader can spark innovation and drive development.
The BootCamp is still in a Beta-Release but I'm proud of the current state. This will help us a lot of getting new recruits on board a lot faster.
Since every section is saved after the completion the recruit can simply resume where he left off.
Another great feature is if we decide to add a new training, everybody can just go through the BootCamp again and just complete the new section. This can be pretty much anything we need at this point.
As I've said, this is just in Beta, but this is going to be a great help to get new people on board. Stay tuned for any updates on this.
You can find the code here: https://github.com/Regimental-Combat-Team-VII/RCT-7-Bootcamp.Stratis
For any questions hit me up on Twitter.