268 lines
9.0 KiB
Markdown
268 lines
9.0 KiB
Markdown
# Original Version (by bram)
|
|
|
|
## Python code + explaination
|
|
|
|
We wanted to make a working connection between our websocket wich runs all the data gatherd by our nodes and a live feed to our database.
|
|
So we set out to make this connection using python.
|
|
|
|
At first we needed to import the folowing librarys:
|
|
|
|
```js
|
|
// everything is running async so the code can run together and not interfere with eachother.
|
|
import asyncio
|
|
import websockets
|
|
// a library to connect to the database.
|
|
import mysql.connector
|
|
import json
|
|
```
|
|
|
|
Then we began the process of connecting with both the websocket and the database.
|
|
|
|
First-off, we began making a connection to the database by using a mysql library in wich we gave the log in in order to connect to our ow database.
|
|
|
|
```js
|
|
//the data that has to be pushed needs to be
|
|
async def process_data(data):
|
|
try:
|
|
mydb = mysql.connector.connect(
|
|
host="localhost",
|
|
user="*****",
|
|
password="*********",
|
|
database="*******"
|
|
)
|
|
```
|
|
|
|
Then after this we code in the infromation we want to put inside of the database.
|
|
The data collected from the websocket is json data, so this has to be changed.
|
|
|
|
```js
|
|
//Making a variable for the database connection
|
|
cursor = mydb.cursor()
|
|
//Making a variable for the database connection
|
|
MACDataReading = mydb.cursor()
|
|
//find the correct section and interact with it.
|
|
MACDataReading.execute("SELECT MAC FROM Node")
|
|
//the query for what needs to be inserted into the database ( atleast where it has to go).
|
|
query = "INSERT INTO `Measurement` (NodeID, Type, Value) VALUES (%s, %s, %s)"
|
|
//the recieved data from the websocket is json data so needs to be changed.
|
|
processedData = json.loads(data)
|
|
//variables about the recieved data points.
|
|
processedTemp = (processedData['Temp'])
|
|
processedHumi = (processedData['Humi'])
|
|
processedECo = (processedData['eCO2'])
|
|
processedTvoc = (processedData['TVOC'])
|
|
processedMAC = (processedData['node'])
|
|
//change the recieved macadress to a tuple.
|
|
MACTuple = (processedMAC,)
|
|
//fetch the data from the database an[d put it in an array.
|
|
MACDataFetching = MACDataReading.fetchall()
|
|
MACArray = list(MACDataFetching)
|
|
|
|
//see if the fetched data is not in the gotten array.
|
|
//otehrwise insert it into the database directly.
|
|
if MACTuple not in MACArray:
|
|
addingNode = "INSERT INTO `Node` (MAC) VALUES (%s)"
|
|
cursor.execute(addingNode, MACTuple)
|
|
mydb.commit()
|
|
//the websocket data that needs to be sent to the database.
|
|
pushingDataArray = [(1, "Temp", processedTemp), (1, "Humi", processedHumi), (1, "eCO2", processedECo), (1, "TVOC", processedTvoc)]
|
|
// forloop, go allong the array.
|
|
for i in pushingDataArray:
|
|
print(query ,i)
|
|
cursor.execute(query, i)
|
|
mydb.commit()
|
|
// show me the error it there is one.
|
|
except mysql.connector.Error as err:
|
|
print("MySQL Error:", err)
|
|
finally:
|
|
cursor.close()
|
|
mydb.close()
|
|
```
|
|
|
|
After fully connecting t othe database, making statements of what to put there and telling the code what to do, we ofcourse need to write the code to connect to the weebsocket.
|
|
We begin by telling our websocket id and what type of port we are using.
|
|
Then we will collect live data from the conected websocket, store it in a variable, and then in the previous code
|
|
|
|
```js
|
|
//here the connection to the websocked is made
|
|
async def receive_data():
|
|
uri = "****************"
|
|
|
|
try:
|
|
async with websockets.connect(uri) as websocket:
|
|
while True:
|
|
// the data collected from the websocket is.
|
|
data = await websocket.recv()
|
|
// data recieved: conformation.
|
|
print(f"Received data: {data}")
|
|
await process_data(data)
|
|
// error sowing.
|
|
except websockets.ConnectionClosedError as e:
|
|
print("WebSocket connection closed:", e)
|
|
|
|
async def main():
|
|
await receive_data()
|
|
|
|
asyncio.run(main())
|
|
```
|
|
|
|
# New Version (by sietse)
|
|
|
|
## Changes made
|
|
|
|
The original code was a good start, but it had some issues. The code could only handle the data from the sensorNodes and didn't include the nodeID for measurements.
|
|
|
|
Since we have 2 kind of nodes (sensorNodes and enqueteNodes) we needed to make another function to commit the enqueteData in the database. I have also made a filter to know which data is from the sensorNodes and which data is from the enqueteNodes. This way we can commit the data to the right table in the database.
|
|
|
|
I have also added a function to get the nodeID from the MAC address. This way we can commit the data to the right node in the database.
|
|
|
|
## The new "filter" code
|
|
|
|
### Function to get a list with macAdresses from the sensorNodes and enqueteNodes
|
|
|
|
To filter i have made 2 lists, one with all the mac adresses of the sensorNodes and the other with the mac adresses of the enqueteNodes.
|
|
|
|
The function that handles that and updates the list is the following:
|
|
|
|
```python
|
|
async def getNodeInfo(type):
|
|
global sensorNodeArray
|
|
global enqueteNodeArray
|
|
|
|
nodeInfoArray = []
|
|
|
|
id = (type,)
|
|
mydb = dbLogin()
|
|
cursor = mydb.cursor()
|
|
cursor.execute("""SELECT MAC FROM Node WHERE Type = %s""", id)
|
|
nodeInfo = cursor.fetchall()
|
|
|
|
for tuples in nodeInfo:
|
|
for item in tuples:
|
|
nodeInfoArray.append(item)
|
|
print(nodeInfoArray)
|
|
|
|
cursor.close()
|
|
mydb.close()
|
|
|
|
if type == 'sensor':
|
|
sensorNodeArray = nodeInfoArray
|
|
print(sensorNodeArray)
|
|
return sensorNodeArray
|
|
|
|
elif type == 'enquete':
|
|
enqueteNodeArray = nodeInfoArray
|
|
return enqueteNodeArray
|
|
```
|
|
|
|
As you can it works like this:
|
|
|
|
1. It gets the MAC adresses from the database with the type of node you want to get the data from. (sensor or enquete)
|
|
|
|
2. It executes the command and puts the data in a list.
|
|
|
|
3. It uses a nested for loop to get the data out of the tuples and puts it in the nodeInfoArray.
|
|
|
|
4. It updates, depending on what type, the sensorNodeArray or the enqueteNodeArray with the new data (NodeInfoArray).
|
|
|
|
5. It returns the array with the data.
|
|
|
|
### The filter code
|
|
|
|
Now that we have the data we can filter the data from the websocket.
|
|
|
|
```python
|
|
data = await websocket.recv()
|
|
|
|
processedData = json.loads(data)
|
|
macAdress = processedData['node']
|
|
|
|
if "Temp" in processedData:
|
|
type = 'sensor'
|
|
else:
|
|
type = 'enquete'
|
|
|
|
await getNodeInfo('sensor')
|
|
await getNodeInfo('enquete')
|
|
|
|
if macAdress in sensorNodeArray:
|
|
nodeID = await getNodeID(macAdress)
|
|
await processSensorNodeData(data, nodeID)
|
|
elif macAdress in enqueteNodeArray:
|
|
nodeID = await getNodeID(macAdress)
|
|
await processEnqueteNodeData(data, nodeID)
|
|
else:
|
|
await newNode(macAdress, type)
|
|
```
|
|
|
|
As you can see its alot of code to explain. So to make it easier i made a mermaid diagram to show how the code works / what it does.
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Get data from websocket] --> B{Is it sensor data or enquete data?}
|
|
B -->|sensor| C[Get sensorNodeArray]
|
|
B -->|enquete| D[Get enqueteNodeArray]
|
|
B -->|New node| E[Add new node to database]
|
|
C -->|data| G[Process sensorNodeData]
|
|
D -->|data| H[Process enqueteNodeData]
|
|
```
|
|
|
|
## The function to get the nodeID
|
|
|
|
This function is used to get the nodeID from the MAC adress. This way we can commit the data with the right id in the database.
|
|
|
|
The function to get the nodeID from the MAC adress is the following:
|
|
|
|
```python
|
|
async def getNodeID(macAdress):
|
|
id = (macAdress,)
|
|
mydb = dbLogin()
|
|
cursor = mydb.cursor()
|
|
cursor.execute("""SELECT nodeID FROM Node WHERE MAC = %s""", id)
|
|
data = cursor.fetchall()
|
|
|
|
for tuples in data:
|
|
for item in tuples:
|
|
nodeID = item
|
|
|
|
return nodeID
|
|
```
|
|
|
|
1. It gets the nodeID from the database with the MAC adress.
|
|
2. It executes the command and puts the data in a list.
|
|
3. It uses a nested for loop to get the data out of the tuples and puts it in the nodeID.
|
|
4. It returns the nodeID.
|
|
|
|
## The function to commit the data from the sensorNodes
|
|
|
|
This function is alot like the original one, with the only 2 changes being that it now also commits the nodeID and that the code to make a new node is now in a different function.
|
|
|
|
[Link to code](server\data.py)
|
|
|
|
## The function to commit the data from the enqueteNodes
|
|
|
|
This function is alot like the sensorNode function. It just commits the data to the enqueteData table in the database. And it has another data.
|
|
|
|
[Link to code](server\data.py)
|
|
|
|
## The function to add a new node to the database
|
|
|
|
This function is used to add a new node to the database. This is used when a new node is connected to the websocket, but not yet in the database.
|
|
|
|
The function to add a new node to the database is the following:
|
|
|
|
```python
|
|
async def newNode(mac, type):
|
|
id = (mac, type)
|
|
mydb = dbLogin()
|
|
|
|
cursor = mydb.cursor()
|
|
cursor.execute("INSERT INTO `Node` (MAC, Type) VALUES (%s, %s)", id)
|
|
print("new node assigned")
|
|
mydb.commit()
|
|
```
|
|
|
|
1. It gets the MAC adress and the type of node from the arguments.
|
|
2. It executes the command to add the new node to the database.
|
|
3. It prints that a new node is assigned.
|
|
4. It commits the data to the database. |