WordPress
WordPress is one of the most successful and important pieces of software on the internet. It’s the web’s leading platform running around 30% of all websites.
WordPress’ strength is as a blogging app and a content management system for semi-technical users, but it is flexible enough to do just about anything.
Being an open source platform with so many users has made WordPress a standard. Whatever you want to do there is probably already a high quality, well tested, regularly updated solution available. Usually in the form of a free plugin that can be implemented quickly and easily with no additional code. This is why WordPress is such a compelling option.
Python
Python is one of the most popular programing languages for websites and the most popular programing language for science, data analytics, fin-tech and web scraping.
This is partly due to Python’s fantastic numbers, math and charting libraries. In the case of web scraping it’s also due to Python’s efficiency and specialized libraries and frameworks such as Beautiful Soup and Scrapy.
Never the twain shall meet…unless!
WordPress and its plugins are coded in PHP. WordPress and Python have nothing to do with each other. You can’t use them together so choose 1. If you want to blog or do content management use WordPress. If you want to do data analytics, fin-tech or web scraping use Python.
But what if you want both of those in the 1 app? For example what if you want to do Python data analyses on the back end then display the results thorough WordPress on the front end? Perhaps embed the results as graphs on blog posts made by non-technical admin?
Well there is a bridge that both these technologies work perfectly with which can link them together into a single app. WordPress loves MySQL databases, Python loves MySQL databases. That’s it! That is how you combine the best of both technologies.
Whatever you want to do with Python put the results into a MySQL database. Then output those results through a custom WordPress plugin. The plugin will make MySQL queries to the database and display the results on the front end.
WordPress database or separate database?
You can use the WordPress site’s own database or you can use any MySQL database separate from the WordPress site. Both options have pros and cons and which you choose will depend on your use case.
WordPress database advantages
The advantages of using the WordPress site’s own database is that you can do everything the WordPress way and everything will work like normal.
Your app will have a simpler architecture and can be hosted on managed WordPress hosts. WordPress and its plugins will all work as expected with the data. For example it will be backed up by your standard backup solution, migrated by standard migration tools and search and replaced by standard search and replace scripts.
At the code level you will be able to use WordPress’ standard database bindings and functions.
For most cases you should keep the data in custom tables in your WordPress site’s database. So when should you not do that? When should you keep the data in a separate MySQL database?
Size matters
Large datasets of over a gigabyte should go in their own database.
This is because the WordPress eco-system has many standard components like plugins, managed hosting, import/export tools, migration tools, search and replace scripts and backup solutions that won’t work with a very large database. Often because they try to act on the entire database with PHP which times out.
If you would still rather keep the very large dataset in the WordPress database there are custom solutions to code your way around these problems but just using a separate database will be easier.
Security concerns
Data kept in your WordPress site’s database can be accessed, modified and deleted by the default WordPress MySQL user.
All 3rd party WordPress plugins and themes get to use this user and therefore can read, modify and delete the data in your custom tables.
It’s not possible to set WordPress’ standard MySQL user with limited permissions on certain tables without modifying WordPress core (which you don’t want to do).
If the data is particularly sensitive keep it in an external MySQL database and have your custom WordPress plugin get the data through a MySQL user with limited permissions for only the required tables.
On that topic, if you are using the WordPress site’s database make sure the python system uses a separate MySQL user with limited permissions for only the required tables when connecting to the database, not the default WordPress MySQL user.
Hosting and connecting the different components
Co-location or local network
The simplest version of this architecture has 2 components; the WordPress site with its database and the python system. For simplicity, performance and security they should be hosted in containers on the same server or separate servers locally networked through a virtual private cloud. This is what that would look like.
If you do use a separate database to the WordPress site’s it could be in the same server or container as the WordPress site. This makes data retrieval fast, however some managed WordPress hosts wont allow this and if the python system is putting a very large load on this database it could effect website performance.
The separate MySQL database could also be in the same server or container as the python system or it could be in a stand alone server or container such as a managed database.
Separate hosting
If co-location and local networks are not possible you can host these components anywhere. They can be on a different server on a different network on a different host in a different country.
In that case you should set up an SSH tunnel between the systems which is more secure than exposing a MySQL port to the open internet for the other system to connect to.
Still this will be more complex, worse for performance and worse for security than co-locating or local networking the components as they will be communicating over the internet, a public network with latency. This is what that would look like with a stand alone database.
At that point you might be better off making an API between the components.
Why not use an API
For some applications an API will be a better option. For some it wont.
For the python system generally performing MySQL transactions will be quicker, easer and more flexible than using WordPress’ built in REST API.
You could have the Python system send the data to the WordPress site via API in a JSON payload or similar. However if the python system was already using MySQL, converting that data to another format like JSON is more data processing.
Also WordPress and its built in functions work well with MySQL out of the box. Using another format delivered via an API will require more custom coding on the WordPress side.
Challenges
Python, MySQL and WordPress are all reliable, secure and performant technologies and they combine very well together in the architecture I have described.
The challenges are due to the fact that Python and WordPress sit far apart on the web development spectrum and are normally used for different types of apps. There is no set formula or documented best practices here. You will have a lot of decisions to make.
Developers
There are very few developers who are good at both WordPress and Python. In fact a lot of pythonistas would turn up their nose at WordPress and deliberately try not to use it. Generally you will have to hire both and hope they work well together.
Hosting options
In order to have the fastest and most secure communication to and from the database you want to have all components running on the same server or privately networked. To do that you might have to forgo serverless architecture or managed hosting and instead self host things on VPSs.
That means someone on the dev team also has to be a system admin, making the chance of finding individual developers who can do it all even less likely.
Products like GridPane and SpinupWP can make self hosting WordPress easier and give some more flexibility.
Conclusion
Despite the challenges, the rewards of using the best possible data processing language with the best possible content management platform in the same app make this a compelling architecture for many applications.
About Eugene Abungana
I have worked with several companies in the past including Economy Watch, and Milkroad. Writing for BitEdge is highly satisfying as I get an opportunity to share my knowledge with a broad community of gamblers.
More by Eugene Abungana