Difference between revisions of "SmartPlug Technical"

From OpenCircuits
Jump to navigation Jump to search
(Created page with "Python Smart Plug Technical = Overview = These notes are here so you can more easily modify the code. Contact me Russ Hensel if you need additional help. Before modify...")
 
 
(22 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
= Overview =
 
= Overview =
 +
== General Information ==
 +
These notes are here so you can more easily modify the code.  Contact me Russ Hensel if you would like additional information.
 +
 +
Before modifying the code it is best to understand how it works. Here is an overview of the general plan, details can be filled out by reading the code.
 +
 +
The "Application" is really two different applications, each with its own main program and GUI, but sharing a lot of the other code.  At one point I though I might merge them into one application, but 2 apps seems better.
 +
The two applications are: SmartPlugGraph ( in smart_plug_graph.py ) and SmartPlug ( in smart_plug.py ).  The apps are very similar so I will explain them in a combined document: generally what is said about SmartPlug also applies to SmartPlugGraph ( the significant difference is that SmartPlugGraph is single threaded, so ignore content, for it, that refers to multi threaded..
  
−
These notes are here so you can more easily modify the code.  Contact me Russ Hensel if you need additional help.
 
  
−
Before modifying the code it is best to understand how it works. Here is an overview of the general plan, details can be filled out by reading the code.
+
The overall architecture is called the model view controller or MVC. The class SmartPlug can be viewed as the main class.  To run the program run its file smart_plug.py ( see code at end of file).  SmartPlug  is the controller in MVC it is responsible for all overall control, it directly or indirectly creates all other program objects.
 +
 
 +
The view component is called GUI ( in gui.py and for SmartPlugGraph is called also GUI but is in GUI_for_graph.py ). It creates all the visible components, and relays user input to the controller.
 +
You can unplug the GUI object from the application and plug in new components. Don't like the GUI? You could modify mine, or you could make a modification and choose which one to use. This is sort of like a skin for an application.  
  
−
The architecture is called the model view controller or MVC. The class SmartTerminal ( in smart_terminal.py ) could be viewed as the main class. To run the program run its file ( see code at end of file)  SmartTerminal is the controller in MVC it is responsible for all overall control, it directly or indirectly creates all other program objects.
+
Two other important components are called Logger ( in logger.py ) and Parameters ( in parameters.py ). The controller creates one of each, and make them available to the other components. The other components can interact with them, and uses them respectively for logging events, and getting access to parameters ( those aspects of the application that are particularly easy to change ). I describe more of this in [[My Python Coding Conventions]]
  
−
The view component is called GUI ( in gui.py ). It creates all the visible components, and relays user input to the controller.
+
The application has a main thread running in a Tkinter mainloop.  There is also a second thread called a "helper" running which makes some processing much easier.  To make GUI mainloop responsive to both the GUI and its own processing it uses a pseudo event loop or a polling subroutine that is implemented in xxxxx. The frequency which polling occurs is set in parameters, the relatively low rate of 100 ms between calls ( .1 sec ) seems to give a perfectly responsive application in most cases. I have run it as fast as once every 10 ms. Have not tried to find a limit.
  
−
The model component is the component that actually does the communication it is called RS232Driver ( in rs232driver.py ) and like the GUI it is controlled by the controller.
+
== Development Environment ==
  
−
The GUI is not allowed to directly communicate with the model and vise versa. Thus you can unplug them from the application and plug in new components. Don't like the GUI? You could modify mine, or you could make a modification and choose which one to use. This is sort of like a skin for an application. You can even set up to run with no GUI at all. The RS232Driver like the GUI easy to remove and replace in the program, its use has been parameterized in to the Parameter object, so to use SPI instead of RS232 all we have to do is write an SPI object and change the values in Parameter.
+
*Windows 10 64bit
 +
*Anaconda Spyder 3.x
 +
*Python 3.7
  
−
Two other important components are called Logger ( in logger.py ) and Parameters ( in parameters.py ). The controller creates one of each, and make them available to the other components. The other components can interact with them, and uses them respectively for logging events, and getting access to parameters ( those aspects of the application that are particularly easy to change ).
+
== Runtime Environments ==
 +
Should Run in:
 +
*Windows, Mac, Linux, Raspberry Pi with desktop support ( not headless )
 +
*Python 3.6 up.
  
−
The application has a main thread running in a Tkinter mainloop.  There is also a second thread called a "helper" running which makes some processing much easier.  To make gui mainloop responsive to both the GUI and its own processing it uses a pseudo event loop or a polling subroutine that is implemented in SmartTerminal.polling().  This is where data is received from there comm port ( or sometimes the responsibility is passed to the helper thread). The frequency which it is called is set in parameters, the relatively low rate of 100 ms between calls ( .1 sec ) seems to give a perfectly responsive application in most cases.  I have run it as fast as once every 10 ms.  Have not tried to find a limit. The second thread is mostly intended for your custom processing, see the section Processing below.
+
I have run in:
 +
Windows 10, Python 3.7
 +
*more coming, or let me know.
  
 
= Components and Functions =
 
= Components and Functions =
  
 
* Tkinter for the GUI
 
* Tkinter for the GUI
−
* pySerial for serial communications
 
 
* pyLogging for logging
 
* pyLogging for logging
 
* SmartTerminal a model-view-controller controller with some modifications  ( '''[https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller Model–view–controller - Wikipedia ]'''  or perhaps closer to '''[https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93presenter Model–view–presenter - Wikipedia ]''' )
 
* SmartTerminal a model-view-controller controller with some modifications  ( '''[https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller Model–view–controller - Wikipedia ]'''  or perhaps closer to '''[https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93presenter Model–view–presenter - Wikipedia ]''' )
 
* Second thread for processing without being blocked by/blocking the GUI
 
* Second thread for processing without being blocked by/blocking the GUI
 
* Global Singleton for app cohesion.  
 
* Global Singleton for app cohesion.  
 +
* Parameter file for easy customization.
 +
* Mathplotlib for graphing.
 +
* SQLLite for the database.
  
−
== The Controller: SmartTerminal ==  
+
== The Controllers for Smart Plug ==  
  
−
   
+
The class for the MVC controller is also the class you create to run the application: see the code at the bottom of the file, just run the file. Similar code is at the bottom of some of the other source files to make it convenient to run from those files, this is just to make it easier in development, the code in smart_plug.py is the model for what should be used.  Sometimes the code at the bottom of other file may have code for testing objects in the file, it may not be maintained, may not work as intended.
  
−
This is the class you create to run the application see the code at the bottom of the file, just run the file.  Similar code is at the bottom of some of the other source files to make it convenient to run from those files, this is just to make it easier in development, the code in smart_terminal is the model for what should be used.  Sometimes the code at the bottom of the file may have code for testing objects in the file, it may not be maintained, may not work as intended.
+
The __init__ method is the initialization and "run" method for the application.  Much of the code that would normally be in it has been moved to .restart which is used to restart the application when just the parameters have been changed.  See the docstring there.
−
 
 
−
T __init__ method is the initialization and "run" method for the application.  Much of the code that would normally be in it has been moved to SmartTerminal.restart which is used to restart the application when just the parameters have been changed.  See the docstring there.
 
  
 
== Global Values ==
 
== Global Values ==
  
 
Yes I know that globals are bad, but they can be useful.  For example many class instances need to access the parameter file.  This can be done using the singleton class AppGlobal.
 
Yes I know that globals are bad, but they can be useful.  For example many class instances need to access the parameter file.  This can be done using the singleton class AppGlobal.
−
It has values at the class level ( not instance ) that are available simply by importing the class.  Most values are defaulted to None, and are set to valid values as the application initializes.
+
It has values at the class level ( not instance ) that are available simply by importing the class.  Most values are originally defaulted to None, and are set to valid values as the application initializes and runs.
  
 
== Threads ==
 
== Threads ==
  
−
The application runs two threads, one for the GUI and one where processing can occur ( HelperThread ) with out locking up the GUI.  There are 2 queues that allow the threads to communicate.
+
The application runs two threads, one for the GUI and one where processing can occur ( HelperThread ) with out locking up the GUI.  There are 2 queues that allow the threads to communicate.  Multithreading is not used in the graphing application as of now.
  
 
=== The Tkinker Thread ===
 
=== The Tkinker Thread ===
Line 50: Line 66:
  
 
=== HelperThread ===
 
=== HelperThread ===
 +
 +
Refers to similar but different program !!  Will update for this program soon.
  
 
HelperThread in smart_terminal_helper.py  This class provides the support for a second thread of execution that does not block the main thread being run by Tinker.  I call the two threads the GUI Thread (gt) and the Helper Thread ( ht ).  It can get confusing keeping track of which method is running in which thread, I sometimes annotate them with gt and ht.  The helper thread is started by running HelperThread.run() which pretty much just runs a polling task in HelperThread.polling().  HelperThread.polling() is an infinite loop, it uses sleep to set the polling rate.  When used with the green house processing module, it may call a function there that is its own infinite loop.  There are a lot of details here, I should write some more about it.
 
HelperThread in smart_terminal_helper.py  This class provides the support for a second thread of execution that does not block the main thread being run by Tinker.  I call the two threads the GUI Thread (gt) and the Helper Thread ( ht ).  It can get confusing keeping track of which method is running in which thread, I sometimes annotate them with gt and ht.  The helper thread is started by running HelperThread.run() which pretty much just runs a polling task in HelperThread.polling().  HelperThread.polling() is an infinite loop, it uses sleep to set the polling rate.  When used with the green house processing module, it may call a function there that is its own infinite loop.  There are a lot of details here, I should write some more about it.
Line 55: Line 73:
 
== Parameters ==
 
== Parameters ==
  
−
Parameters ( in parameters.py ) this is pretty much a structure ( that is all instance variables ) that is instantiated early in the life of the application.  It passes values, strings, numbers, objects around the application to other parts of the application that need them.  Much of the appearance and behavior of the application is controlled here.
+
Parameters ( in parameters.py ) this is pretty much a structure ( that is all instance variables ) that is instantiated early in the life of the application.  It passes values, strings, numbers, objects to application elements that need them.  The instance of parameters is made globally available thru the AppGlobal class. Values in Parameters are treated as constants, read only.  Much of the appearance and behavior of the application is controlled here.
  
 
The standard gui has a button to kick off editing of this file, the application may then be restarted ( another button ) with the new values.
 
The standard gui has a button to kick off editing of this file, the application may then be restarted ( another button ) with the new values.
  
−
There are a couple of meta parameters, the most important of which is mode which is then used in conditionals later in parameters.  Except for this sort of thing there is really not much "code" in parameters.  You can change this code pretty much as much as you like, as long as you end up setting up values for the required parameters.
+
There are a couple of meta parameters, including os_win, mode and computername which then may be used in conditionals later in parameters.py.  Except for this sort of thing there is really not much "code" in parameters.  You can change this code pretty much as much as you like, as long as you end up setting up values for the required parameters.
  
 
The code is extensively commented: use that for documentation.
 
The code is extensively commented: use that for documentation.
Line 65: Line 83:
 
Values are often set to None as a default, then later set to some other value.  Or the value may be set several times in a row ( this is an artifact of messing with the values ); only the last value set has any meaning.
 
Values are often set to None as a default, then later set to some other value.  Or the value may be set several times in a row ( this is an artifact of messing with the values ); only the last value set has any meaning.
  
−
If asked for in the command line you can also envoke a second parameter file. This is handy if you want two different instances of the terminal.
+
For more info see: [[Configuration Files For Python]]
  
−
For a lot more info see:  [[Smart Terminal Parameter Examples]]
+
== DataBase ==
  
−
== Processing Modules==
+
The database is SQLLite.  So far it seems adequate to the task.  Access is via string of SQL and variables bound to those statements.  Look for the code in .......
−
 
 
−
So called processing modules offer you the opportunity to put some custom automatic processing in your terminal.  Typically this is matched up with a custom microcontroller application on the other end of the com port.  These are all implemented as descandants of the class ABCProcessing.  ( so far ABCProcessing does not add much value ) A couple of examples of processing modules and the arduino code are included in the download package, but beware they are in various states of maintenance. See: [[Writing You Own Extensions to SmartTerminal]]
 
−
 
 
−
 
  
 
== Polling ==
 
== Polling ==
−
Both threads have method that perform polling for events, events like received text, or items in their queue that may have been sent from the other thread.  More info in [[Python Smart Terminal Technical Details]]
+
Both threads have method that perform polling for events often for items in their queue that may have been sent from the other thread.  Info on a similar app in [[Python Smart Terminal Technical Details]].
  
 
== Logging ==
 
== Logging ==
−
This uses the standard Python logging class.  Logging level and other logging details are set up using the parameter file.
+
This uses the standard Python logging class.  Logging level and other logging details are controlled using the parameter file.
  
 
== Other Classes ==
 
== Other Classes ==
−
For now the documentation, as far as it exists is in the source code.
+
For now the documentation, as far as it exists is in the source code.  This probably will not change.
  
−
= Coding Conventions Etc. =
+
See:  [[My Python Coding Conventions]]
  
−
In reading the code it may be of some use to know what conventions I have ( tried ) to follow. The code has been developed over quite a period of time so the standards are not uniform.  What I write here are the standards that are in quite a bit of the code and the directions that I am trying to move.  In all of the coding consistency is an important standard.  Here are some areas.
+
= Coding Conventions =
 +
See: [[My Python Coding Conventions]]
  
−
== Names ==
+
= See Also =
−
Be consistent: this is good but have not been very successful in standards: I keep changing my mind.  Names across classes are pretty consistent.  I am avoiding short names and try to make them descriptive enough that they are somewhat self documenting.  References are often copied across objects for easy access ( lots of parameters for example ); when this happens the name of the object is generally ( should be always ) the same in both objects.
 
−
 
 
−
== Formatting ==
 
  
−
Nothing special here but I like white space and use a lot.  This is not standard Python.  But this is what I like.
+
Master Page for this Project:
 +
[[Python Control of Smart Plugs]]
  
−
== Docstrings ==
 
−
Work towards using them.  Not good as of 2017 Jan
 
−
 
−
== Imports ==
 
−
*In most cases use the format "import xyz" so the name space is not polluted and so it is easy to identify just what an imported class is.
 
−
*In in objects that are almost all GUI then using "from Tkinter import *"  is ok but better is: "import Tkinter as Tk"
 
−
*I normally have only one or a few classes in a file so there is a lot of what I call "local imports". 
 
−
*Almost all imports are at the top of a file, std library imports first then "local imports".
 
−
 
−
== Object Orientation ==
 
−
 
−
Almost everything is a class.  Not much in the way of module functions, not many classes in a module.  I am trying to have all my classes descend from something be it only Object.  And now that I am in Python 3.6 this is how it always works.
 
−
 
−
== Documentation for Class Instance Methods ==
 
−
 
−
Look something like this:
 
−
 
−
    def create_class_from_strings( self, module_name, class_name):
 
−
        """
 
−
        This will load a class from string names
 
−
        It makes it easier to specify classes in the parameter file.
 
−
        I believe it is used for both the comm drive and the "processor"
 
−
        args:  strings
 
−
        ret:  instance of the class
 
−
        Side effects Class created
 
−
        """
 
−
 
−
The comment should give the intent of the method, some hint as to the args ( which hopefully have good names ), and some info. on the return value.  zip means nothing, void....
 
−
 
−
I am moving toward using __ and _ as prefixes for more private methods, but have not gone too far in this direction.
 
−
 
−
 
−
 
−
= See Also =
 
  
 +
== A Related Python Program==
 
*[[Python Smart Terminal Technical Details]] More detailed technical issues.
 
*[[Python Smart Terminal Technical Details]] More detailed technical issues.
 
*[[Smart Terminal GUI]] how the gui works and interfaces the user.
 
*[[Smart Terminal GUI]] how the gui works and interfaces the user.
Line 135: Line 115:
 
* and the categories below may be useful ( click on them ).
 
* and the categories below may be useful ( click on them ).
  
−
[[Category:Python SmartPlug]]  [category:Python]]
+
 
 +
 
 +
[[Category:Python SmartPlug]]  [[Category:Python]]

Latest revision as of 14:32, 19 September 2019

Python Smart Plug Technical

Overview[edit]

General Information[edit]

These notes are here so you can more easily modify the code. Contact me Russ Hensel if you would like additional information.

Before modifying the code it is best to understand how it works. Here is an overview of the general plan, details can be filled out by reading the code.

The "Application" is really two different applications, each with its own main program and GUI, but sharing a lot of the other code. At one point I though I might merge them into one application, but 2 apps seems better. The two applications are: SmartPlugGraph ( in smart_plug_graph.py ) and SmartPlug ( in smart_plug.py ). The apps are very similar so I will explain them in a combined document: generally what is said about SmartPlug also applies to SmartPlugGraph ( the significant difference is that SmartPlugGraph is single threaded, so ignore content, for it, that refers to multi threaded..


The overall architecture is called the model view controller or MVC. The class SmartPlug can be viewed as the main class. To run the program run its file smart_plug.py ( see code at end of file). SmartPlug is the controller in MVC it is responsible for all overall control, it directly or indirectly creates all other program objects.

The view component is called GUI ( in gui.py and for SmartPlugGraph is called also GUI but is in GUI_for_graph.py ). It creates all the visible components, and relays user input to the controller. You can unplug the GUI object from the application and plug in new components. Don't like the GUI? You could modify mine, or you could make a modification and choose which one to use. This is sort of like a skin for an application.

Two other important components are called Logger ( in logger.py ) and Parameters ( in parameters.py ). The controller creates one of each, and make them available to the other components. The other components can interact with them, and uses them respectively for logging events, and getting access to parameters ( those aspects of the application that are particularly easy to change ). I describe more of this in My Python Coding Conventions

The application has a main thread running in a Tkinter mainloop. There is also a second thread called a "helper" running which makes some processing much easier. To make GUI mainloop responsive to both the GUI and its own processing it uses a pseudo event loop or a polling subroutine that is implemented in xxxxx. The frequency which polling occurs is set in parameters, the relatively low rate of 100 ms between calls ( .1 sec ) seems to give a perfectly responsive application in most cases. I have run it as fast as once every 10 ms. Have not tried to find a limit.

Development Environment[edit]

  • Windows 10 64bit
  • Anaconda Spyder 3.x
  • Python 3.7

Runtime Environments[edit]

Should Run in:

  • Windows, Mac, Linux, Raspberry Pi with desktop support ( not headless )
  • Python 3.6 up.

I have run in: Windows 10, Python 3.7

  • more coming, or let me know.

Components and Functions[edit]

  • Tkinter for the GUI
  • pyLogging for logging
  • SmartTerminal a model-view-controller controller with some modifications ( Model–view–controller - Wikipedia or perhaps closer to Model–view–presenter - Wikipedia )
  • Second thread for processing without being blocked by/blocking the GUI
  • Global Singleton for app cohesion.
  • Parameter file for easy customization.
  • Mathplotlib for graphing.
  • SQLLite for the database.

The Controllers for Smart Plug[edit]

The class for the MVC controller is also the class you create to run the application: see the code at the bottom of the file, just run the file. Similar code is at the bottom of some of the other source files to make it convenient to run from those files, this is just to make it easier in development, the code in smart_plug.py is the model for what should be used. Sometimes the code at the bottom of other file may have code for testing objects in the file, it may not be maintained, may not work as intended.

The __init__ method is the initialization and "run" method for the application. Much of the code that would normally be in it has been moved to .restart which is used to restart the application when just the parameters have been changed. See the docstring there.

Global Values[edit]

Yes I know that globals are bad, but they can be useful. For example many class instances need to access the parameter file. This can be done using the singleton class AppGlobal. It has values at the class level ( not instance ) that are available simply by importing the class. Most values are originally defaulted to None, and are set to valid values as the application initializes and runs.

Threads[edit]

The application runs two threads, one for the GUI and one where processing can occur ( HelperThread ) with out locking up the GUI. There are 2 queues that allow the threads to communicate. Multithreading is not used in the graphing application as of now.

The Tkinker Thread[edit]

Once Tkinker is started it runs its own mainloop. In order to receive however we need to check the rs232 port from time to time. This is done in SmartTerminal.polling() I call this thread the GUI thread or gt. The terminal is configured to have a second thread called the Helper Thread or ht. In some cases the receiving of data is passed to the Helper Thread so where data is processed in addition to being posted to the GUI. See the section HelperThread for more info.

HelperThread[edit]

Refers to similar but different program !! Will update for this program soon.

HelperThread in smart_terminal_helper.py This class provides the support for a second thread of execution that does not block the main thread being run by Tinker. I call the two threads the GUI Thread (gt) and the Helper Thread ( ht ). It can get confusing keeping track of which method is running in which thread, I sometimes annotate them with gt and ht. The helper thread is started by running HelperThread.run() which pretty much just runs a polling task in HelperThread.polling(). HelperThread.polling() is an infinite loop, it uses sleep to set the polling rate. When used with the green house processing module, it may call a function there that is its own infinite loop. There are a lot of details here, I should write some more about it.

Parameters[edit]

Parameters ( in parameters.py ) this is pretty much a structure ( that is all instance variables ) that is instantiated early in the life of the application. It passes values, strings, numbers, objects to application elements that need them. The instance of parameters is made globally available thru the AppGlobal class. Values in Parameters are treated as constants, read only. Much of the appearance and behavior of the application is controlled here.

The standard gui has a button to kick off editing of this file, the application may then be restarted ( another button ) with the new values.

There are a couple of meta parameters, including os_win, mode and computername which then may be used in conditionals later in parameters.py. Except for this sort of thing there is really not much "code" in parameters. You can change this code pretty much as much as you like, as long as you end up setting up values for the required parameters.

The code is extensively commented: use that for documentation.

Values are often set to None as a default, then later set to some other value. Or the value may be set several times in a row ( this is an artifact of messing with the values ); only the last value set has any meaning.

For more info see: Configuration Files For Python

DataBase[edit]

The database is SQLLite. So far it seems adequate to the task. Access is via string of SQL and variables bound to those statements. Look for the code in .......

Polling[edit]

Both threads have method that perform polling for events often for items in their queue that may have been sent from the other thread. Info on a similar app in Python Smart Terminal Technical Details.

Logging[edit]

This uses the standard Python logging class. Logging level and other logging details are controlled using the parameter file.

Other Classes[edit]

For now the documentation, as far as it exists is in the source code. This probably will not change.

See: My Python Coding Conventions

Coding Conventions[edit]

See: My Python Coding Conventions

See Also[edit]

Master Page for this Project: Python Control of Smart Plugs


A Related Python Program[edit]

Retrieved from "http://www.opencircuits.com/index.php?title=SmartPlug_Technical&oldid=81863"