Today we are returning to Prestashop modules programming. What game shall we play today?

This is the 3rd part of exploration of pandas package.

Part 2: https://www.vallka.com/blog/pandas-in-the-pandaemic-covid-19-in-scotland-statistics-part-2/

Part 1: https://www.vallka.com/blog/leaning-pandas-data-manipulation-and-visualization-covid-19-in-scotland-statistics/

All data are taken from the official websate: weekly-deaths-by-location-age-sex.xlsx

Taken from official the site:

https://www.nrscotland.gov.uk/statistics-and-data/statistics/statistics-by-theme/vital-events/general-publications/weekly-and-monthly-data-on-births-and-deaths/deaths-involving-coronavirus-covid-19-in-scotland/related-statistics

Let's have another look at the graph from the previous part: https://colab.research.google.com/drive/1a5FyWN5psehoqnAUEev0ODE2KNMiFTTE?usp=sharing

New data arrived. The line for the last week is going up a little, but still well below the average.

This is the 2nd part of exploration of pandas package. Part 1 can be found here: https://www.vallka.com/blog/leaning-pandas-data-manipulation-and-visualization-covid-19-in-scotland-statistics/

All data are taken from the official websate:

weekly-deaths-by-location-age-sex.xlsx

Taken from official the site:

https://www.nrscotland.gov.uk/statistics-and-data/statistics/statistics-by-theme/vital-events/general-publications/weekly-and-monthly-data-on-births-and-deaths/deaths-involving-coronavirus-covid-19-in-scotland/related-statistics

Let's quickly repeat data load process from the Part 1:

import numpy as np
import pandas as pd

data_2021 = pd.read_excel ('https://www.nrscotland.gov.uk/files//statistics/covid19/weekly-deaths-by-location-age-sex.xlsx',
 sheet_name='Data',
        skiprows=4,
        skipfooter=2,
        usecols='A:F',
        header=None,
        names=['week','location','sex','age','cause','deaths']
        )

data_2021['year'] = data_2021.week.str.slice(0,2).astype(int)
data_2021['week'] = data_2021.week.str.slice(3,5).astype(int)

data_1519 = pd.read_excel ('https://www.nrscotland.gov.uk/files//statistics/covid19/weekly-deaths-by-location-age-group-sex-15-19.xlsx',
        sheet_name='Data',
        skiprows=4,
        skipfooter=2,
        usecols='A:F',
        header=None,
        names=['year','week','location','sex','age','deaths'])

data_1519['cause'] = 'Pre-COVID-19'

data=data_1519.copy()
data=data_1519.append(data_2021,ignore_index=True)
data

One note here. In part 1 I was using a simple assignment od DateFrame for making a copy. This is wrong. Simple assignment does not create a copy of a DataFrame, it just creates another reference to it, so all changes to 2nd DataFrame affects the original one. To make things even more interesting, pandas provides two versions of copy() function - copy(deep=False) and copy(deep=True) deep=True is default. Here is some discussion about all three.

It is not clear for me now what is the difference between simple assignment and shallow copy() (deep=False), nor my results confirm this discussion. But deep copy() (with default parameter deep=True) seems to be working as expected all the time.

Let's see the original data_1519 to ensure we didn't modify it accidently:

data_1519

Let's quickly create totals

totals = data.groupby(['year','week']).agg({'deaths': np.sum})
totals.loc[15,'deaths_15']=totals['deaths']
totals.loc[16,'deaths_16']=totals['deaths']
totals.loc[17,'deaths_17']=totals['deaths']
totals.loc[18,'deaths_18']=totals['deaths']
totals.loc[19,'deaths_19']=totals['deaths']
totals.loc[20,'deaths_20']=totals['deaths']
totals.loc[21,'deaths_21']=totals['deaths']
totals

Let's do a little bit of pandas. Pandas is (or are? :) extremely popular. Let's just dive in.

First of all, using pandas is the easiest way to open an Excel file. Just one line of code. Let's take, for example, this file -

Weekly deaths by location of death, age group, sex and cause, 2020 and 2021
(10 March 2021)

weekly-deaths-by-location-age-sex.xlsx

Taken from official the site:

https://www.nrscotland.gov.uk/.../related-statistics

import pandas as pd

data = pd.read_excel ('https://www.nrscotland.gov.uk/files//statistics/covid19/weekly-deaths-by-location-age-sex.xlsx',
 sheet_name='Data',
        skiprows=4,
        skipfooter=2,
        usecols='A:F',
        header=None,
        names=['week','location','sex','age','cause','deaths']
        )

data

What is the idea?

Let's start programming Prestashop modules. Where to start from? Here is the documentation:

https://devdocs.prestashop.com/1.7/modules/

Ok. But what would be our first module? It should be very simple, and yet it should be useful. Here is the idea: add BCC to all outgoing emails, that we will have copies of all outgoing emails in out mailbox. Why would we want it? Actually this is what many people want, just try to google. For newly set up website it may be useful. You may want to see how actually the emails look like. And find all versions of emails which Prestashop sends to your customers. Probably after looking to these emails you would want to change them a bit, maybe remove a default "Powered by Prestashop" line at the bottom... You may want to keep monitoring outgoing emails to ensure that next update of Prestashop do not revert your custom emails to default ones (you probably forgot to check "Keep email templates" while updating Prestashop). Ok, let's do it.

Among Prestashop hooks there is one which can be used for our purpose: actionEmailSendBefore

actionEmailSendBefore

Before sending an email This hook is used to filter the content 
or the metadata of an email before sending it or even prevent its sending

Located in: /classes/Mail.php

https://devdocs.prestashop.com/1.7/modules/concepts/hooks/list-of-hooks/

If we have a chance to look inside /classes/Mail.php file, we will see that all the parameters are prefixed with "&", what means we can change them:

$hookBeforeEmailResult = Hook::exec(
        'actionEmailSendBefore',
        [
            'idLang' => &$idLang,
            'template' => &$template,
            'subject' => &$subject,
            'templateVars' => &$templateVars,
            'to' => &$to,
            'toName' => &$toName,
            'from' => &$from,
            'fromName' => &$fromName,
            'fileAttachment' => &$fileAttachment,
            'mode_smtp' => &$mode_smtp,
            'templatePath' => &$templatePath,
            'die' => &$die,
            'idShop' => &$idShop,
            'bcc' => &$bcc,
            'replyTo' => &$replyTo,
        ],
        null,
        true
    );

I couldn't find any place in Prestashop Back Office which could potentially use BCC. So we can assume (for simplicity) it is always empty and we can just set it to our own value in the hook.

Let's create a module!

Luckily, there is a simple way to create a module skeleton. There is a module generator supplied by Prestashop itself:

https://validator.prestashop.com/generator

I wouldn't say it works perfectly but it works. Probably, it is not updated as ofter as new Prestashop versions are released, and contains some bugs. Also, as mentioned in the documentation, if you want to create a Payments module for Prestashop 1.7 you should not use this generator. But for our purpose it as just invaluable.

So put some values to start a module - name, version, author name etc.: