Forem: Valentine

Google Analytics 4 (GA4) - querying raw data with SQL

Valentine — Sat, 11 Mar 2023 13:55:29 +0000

Google Analytics 4 (GA4) is the newest version of Google Analytics, designed to provide businesses with a more comprehensive view of customer interactions across multiple platforms and devices. GA4 provides businesses with valuable insights that can help improve their online presence and increase conversions.

While GA4 reports can be confusing for some, it is possible to query raw data using SQL to gain insights that are relevant to your business. In this article, we will discuss how to use GA4 and Google BigQuery to query raw data and gain valuable insights.

Connecting Google Analytics 4 to Google BigQuery

The first step in using GA4 to query raw data is to connect it to Google BigQuery. This can be done easily by following these steps:

Create a new BigQuery project or use an existing one.
In the GA4 admin panel, select the Data Streams tab.
Select the data stream that you want to export to BigQuery and click on the BigQuery Export tab.
Click on the Link to BigQuery button and follow the prompts to link your GA4 account to your BigQuery project.

Once your GA4 account is linked to your BigQuery project, you can begin querying raw data using SQL.

Querying Raw Data using SQL

One of the most powerful features of GA4 is its ability to export raw data to BigQuery. This allows you to create custom queries using SQL and gain insights that are not available in GA4 reports.

For example, if you want to know the details of users who made purchases during this year, you can use the following SQL query:

 SELECT 
  event_date, 
  event_timestamp, 
  p.value.string_value AS order_id, 
  device.mobile_model_name, 
  device.operating_system_version, 
  traffic_source.name AS campaign, 
  traffic_source.medium, 
  traffic_source.source  
 FROM 
  `ga4-gellifique-uk.analytics_287630133.events_*`, 
  `ga4-gellifique-uk.analytics_287630133.events_*`.event_params p 
 WHERE 
  event_name='purchase' 
  AND key='transaction_id' 
  AND event_date>='20230101' 
 ORDER BY 
  event_timestamp 
 LIMIT 
  1000

In this SQL query, we are selecting the event_date, event_timestamp, transaction_id, mobile_model_name, operating_system_version, name, medium, and source from the raw data. We are also filtering the results to only show events with the event name "purchase" and a transaction ID, and only events that occurred after January 1st, 2023.

Here is the result:

Next SQL query provides an example of how to use Google Analytics 4 (GA4) and Google BigQuery to gain insights into the efficiency of your campaigns based on the number of purchases.

In this query, we are first selecting the event_date, event_timestamp, transaction_id, mobile_model_name, operating_system_version, name, medium, and source from the raw data. We are also filtering the results to only show events with the event name "purchase" and a transaction ID, and only events that occurred after January 1st, 2023.

Next, we are using a subquery to count the number of purchases for each campaign. We are grouping the results by campaign name and ordering the results by the count of purchases in descending order.

SELECT 
  name AS campaign, 
  COUNT(name) AS count 
FROM 
  (
    SELECT 
      event_date, 
      event_timestamp, 
      p.value.string_value AS transaction_id, 
      device.mobile_model_name, 
      device.operating_system_version, 
      traffic_source.name, 
      traffic_source.medium, 
      traffic_source.source 
    FROM 
      `ga4-gellifique-uk.analytics_287630133.events_*`, 
      `ga4-gellifique-uk.analytics_287630133.events_*`.event_params p 
    WHERE 
      event_name='purchase' 
      AND key='transaction_id' 
      AND event_date>='20230101' 
    ORDER BY 
      event_timestamp 
    LIMIT 
      1000
  )
GROUP BY 
  name 
ORDER BY 
  count(name) DESC

Here is the result:

By using this query, you can gain insights into which campaigns are driving the most purchases and adjust your marketing strategies accordingly.

Conclusion

While GA4 reports can be overwhelming, connecting GA4 to Google BigQuery and querying raw data using SQL can provide valuable insights for your business. By using custom queries, you can gain insights that are relevant to your business and help you make data-driven decisions.

textbanner - a module for Prestashop

Valentine — Wed, 22 Feb 2023 14:20:33 +0000

This is a Prestashop module that displays a text banner on the store front-end instead of using an image banner. It is much easier and practical to use simple HTML text with predefined colors. The module was developed by Vallka and is released under the Academic Free License (AFL 3.0).

The module extends the Module class and implements the WidgetInterface. It also has a private variable $write_log that can be used for logging.

The constructor sets the basic properties of the module such as name, tab, version, author, etc. It also sets the display name and description of the module that appears on the back office. Additionally, the minimum and maximum versions of Prestashop with which the module is compatible are set.

The install() method registers the displayBanner, actionObjectLanguageAddAfter, and backOfficeHeader hooks and installs fixtures. The fixtures are used to populate the TEXTBANNER_LINK and TEXTBANNER_DESC configuration variables. The backOfficeHeader() hook is used to add a JavaScript file to the back office, and the actionObjectLanguageAddAfter() hook is used to install the fixtures for each language.

The uninstall() method removes the TEXTBANNER_LINK and TEXTBANNER_DESC configuration variables when the module is uninstalled.

The postProcess() method is used to update the TEXTBANNER_LINK and TEXTBANNER_DESC configuration variables when the user saves the settings. The method updates the values for each language and clears the cache.

The hookBackOfficeHeader() method in the textbanner module is responsible for showing all input fields for all enabled languages simultaneously in the back office. Specifically, it adds the back.js file to the back office header when the configure parameter is set to the name of the textbanner module.

The purpose of back.js is to show all input fields for all enabled languages at once, on the same screen. This helps to remind the administrator to update all languages when making an update, and saves time by not having to switch between different languages to make the same change.

To achieve this, back.js simply changes the visibility of all input fields for all enabled languages to 'show'. This feature improves the user experience for administrators by providing a more efficient and convenient way to manage input fields for multiple languages.

Enjoy on GitHub!

Social links for customers in Prestashop

Valentine — Tue, 21 Feb 2023 20:13:21 +0000

PrestaShop 1.7 Module

This is a module for PrestaShop, an open-source eCommerce platform, that adds extra fields to the customer registration and account pages. The module adds fields for social media account information and preferred communication method.

The module is licensed under the Academic Free License (AFL 3.0) and was developed by vallka. The module's version is 1.0.0, and it is compatible with PrestaShop 1.7 and above.

The module installs a new database table named extra_customer_fields to store the additional information. The module also registers several hooks to integrate with the customer registration and account pages. The additionalCustomerFormFields hook adds the extra fields to the customer registration form. The validateCustomerFormFields hook validates the extra fields before saving them to the database. The actionObjectCustomerUpdateAfter and actionObjectCustomerAddAfter hooks update and add the extra fields to the database when a customer is updated or added. Finally, the actionCustomerFormBuilderModifier hook modifies the customer account form to include the extra fields.

The module has two methods for reading and writing the extra field values: readModuleValues and writeModuleValues. The readModuleValues method reads the extra field values from the database for a given customer ID, and the writeModuleValues method writes the extra field values to the database for a given customer ID.

Here is the link to GitHub - feel free to use!

Prestashop and ImageKit

Valentine — Fri, 11 Jun 2021 20:20:30 +0000

Recently we found a really useful service - imagekit.io.

Image CDN with automatic optimization, real-time transformation, and storage that you can integrate with existing setup in minutes.

What is the most amazing thing about this service, it has a free plan with very a generous set of features and allowances.

Let's use it for a Prestashop theme.

First of all, we need to create an account on imagekit.io. After verifying your email, you'll be able to login to your account. At the very first step you will be asked for IMAGEKIT ID. By default this is a random string value. It make sense to change this random string to something meaningful, like the main portion of your website domain name. After setting all up, you will be unable to change your IMAGEKIT ID, so please do it right now.

Next, the imagekit terminology becomes a somewhat confusing. We need to "Attach existing origin in ImageKit.io". Click on the Attach existing origin in ImageKit.io link and then click on ADD NEW ORIGIN button. Origin Name may be any string, for Origin Type select Web Folder - HTTP(S) server and Magento, Shopify, Wordpress, etc.. For Base URL use base URL, like https://www.example.com (without /images subfolder as it advise you). Leave the bottom checkboxes unchecked (unless you know what they are), and click Submit.

What has confused me a little - where can you find this newly created "origin" after it has been set up? Under "URL endpoins" menu at the left. You will find "Default URL endpoint" there. Click on it, here you can edit your "origins". You might need a new "origin", if you are using a multi-store feature of Prestashop. For example, we have 2 shops and 2 domains for these shops in our Prestashop - gellifique.co.uk for UK shop and gellifique.eu for Spanish shop. Both domains should be added to Imagekit Default URL endpoins as 2 separate "origins". One endpoint, two origins. This was somewhat confusing.

Basically, all done. Now we can use ImageKit URLs instead of original URLs for images in your theme. I wanted to pinpoint all templates in Classic theme which needs to be updated, but as it happened, the Classic theme has changed a lot since we used it to create a custom theme for our webshop. In our case, I had to modify these files (under /themes/mytheme/ folder):

templates/catalog/listing/category.tpl 
templates/catalog/_partials/product-cover-thumbnails.tpl 
templates/catalog/_partials/product-images-modal.tpl 
templates/catalog/_partials/miniatures/product.tpl 
modules/ps_imageslider/views/templates/hook/slider.tpl

In the latest version of the Classic theme some of these files became split into two or more files and some references may have moved somewhere else... So I'll show an example of what I did to my, probably outdated version. Let's take this file:

templates/catalog/_partials/miniatures/product.tpl

It looks like it didn't change much. Let's find a reference to a product image (lines 33-37):

<img
  src="{$product.cover.bySize.home_default.url}"
  alt="{if !empty($product.cover.legend)}{$product.cover.legend}{else}{$product.name|truncate:30:'...'}{/if}"
  data-full-size-image-url="{$product.cover.large.url}"
/>

We need to change src attribute - with what? PHP parse_url() function will help us:

src = "https://ik.imagekit.io/ourimagekitid{parse_url($product.cover.bySize.home_default.url, PHP_URL_PATH)}"

where ourimagekitid is the IMAGEKIT ID we have set up earlier. If we know the exact size of the image here, we may specify it for the Imagekit, for example:

src = "https://ik.imagekit.io/ourimagekitid/tr:w-250,h-250{parse_url($product.cover.bySize.home_default.url, PHP_URL_PATH)}"

Do similar exercises to remaining templates, and the task is completed! However...

We had to hardcode IMAGEKIT ID into several files. Not good. Next issue - sometimes you may want to switch off ImageKit completely, for example in development environment. And the last one (at least for me) - although ImageKit gives us an ability to Purge cache, we have to specify it file by file, and even so it doesn't occure immediately. Sometimes you have to wait a few minutes before it have an effect... Would be nice to implement some feature like file versioning, like attaching ? to the end of the URL.

So let's write another Prestashop module.

Like before, we will create a module skeleton, using (see my first article):

https://validator.prestashop.com/generator

We will register two hooks:

public function install()
{
    Configuration::updateValue('IMAGEKIT_LIVE_MODE', false);
    return parent::install() &&
            $this->registerHook('actionDispatcher') && 
            $this->registerHook('backOfficeHeader');
}

"backOfficeHeader" hook is needed only for attaching some js file to back office (not really important):

public function hookBackOfficeHeader()
{
    if (Tools::getValue('configure') == $this->name || Tools::getValue('module_name') == $this->name) {
        $this->context->controller->addJS($this->_path.'views/js/back.js');
    }
}

And the important one is hookActionDispatcher. It allows us to define a custom plugin for Smarty templates. This plugin can be used as a function in a Smarty template

public function hookActionDispatcher(){
    $this->context->smarty->registerPlugin('function', 'imagekit', array('Imagekit', 'imagekitUrl'));
}

And the function implementation construct the needed URL for ImageKit image:

public static function imagekitUrl($params,$smarty) {
    if ($params['url']) {
        if ( Configuration::get('IMAGEKIT_LIVE_MODE', false) && Configuration::get('IMAGEKIT_ENDPOINT', '')) {
            $tr = $params['tr'] ? 'tr:'.$params['tr'] : '';
            $v = $params['v'] ? '?'.Configuration::get('IMAGEKIT_TS', '') : '';
              return Configuration::get('IMAGEKIT_ENDPOINT', '') . $tr . parse_url($params['url'],PHP_URL_PATH) . $v;
        }

        return $params['url'];
    }
    return '';
}

It takes configuration parameters from the module configuration. IMAGEKIT_LIVE_MODE allows us to switch using ImageKit on/off with a single click. Parameters passed to the function are "url", "tr", and "v" (only "url" parameter is mandatory). "tr" set required transformation in ImageKit format, and v=1 (0 by default) append timestamp "?" querystring to the URL.

So now the src attribute in templates/catalog/_partials/miniatures/product.tpl template will look like this:

src = "{imagekit tr='w-250,h-250' url=$product.cover.bySize.home_default.url}"

What about "v" (versioning) parameter and why all the buzz? As I found out, the only place where it makes sense is templates/catalog/listing/category.tpl template. This is the only place where a newer image is being saved by the same name as a previous one. If you upload a new image for a category with id 25, for example, the image URL will be /img/c/25.jpg (if you upload a jpg file). If you upload a new image, it still will have this name. Thus ImageKit will think it is the same image. You can Purge cache from ImageKit dashboard (providing you know the exact URL), wait some time and the change will happen... But for me it was easier to introduce "versioning". You have to go to my module configuration, double-click on Version (Timestamp) field, ans save configuration. And image URL:

/img/c/25.jpg

will become something like this

/img/c/25.jpg?1623440107030

and ImageKit will treat it as a new image.

In all other templates Prestashop generates a unique image name, using either the actual file name, or some random values, and such problem doesn't raise.

So in the file templates/catalog/listing/category.tpl

the line

<img class="replace-2x" 
    src="{$link->getCatImageLink($subcategory.link_rewrite,$subcategory.id_image,'')}"
    alt="{$subcategory.name|escape:'html':'UTF-8'}"/>

became

<img class="replace-2x" 
    src="{imagekit tr='w-230,h-230' v=1 url=$link->getCatImageLink($subcategory.link_rewrite,$subcategory.id_image,'')}"
    alt="{$subcategory.name|escape:'html':'UTF-8'}"/>

I cannot find a similar line in the modern version of Classic theme. Maybe subcategories images are not shown at all in the modern Classic theme?

Ok, everything is working now as I wanted it. Except for a small problem what I couldn't solve. Maybe some of the readers will point me to a right direction ?

If, for some reason, I will unregister or disable my module, all images will go! Because Smarty plugin "imagekit" doens't exist any more. I wanted to do something like this in my templates:

<img class="replace-2x" 
        {if function_exists("imagekitUrl")}
            src="{imagekit tr='w-230,h-230' v=1 url=$link->getCatImageLink($subcategory.link_rewrite,$subcategory.id_image,'')}"
        {else}
            src="{$link->getCatImageLink($subcategory.link_rewrite,$subcategory.id_image,'')}"
     {/if}
         />

but for some reason it doesn't seem to work. Anyone can help? (Maybe I had to use "modifer" instead of "function" in Smarty plugin, it would be easier than?)

As usual, the source of the module is on GitHub:

https://github.com/vallka/prestamodule_imagekit.git

Laybuy against Prestashop

Valentine — Sat, 22 May 2021 19:40:14 +0000

Recently we subscribed to Laybuy with our Prestashop-based eCommerce website. It provides Prestashop module out of a box. The module is easy to install and setup. However, immediately we have found a problem. In certain circumstances Laybuy module reported an error - totals didn't sum up:

What is the problem? Enabling a debug mode in Laybuy module (thank you developers!) shows the source of the problem: Laybuy adds a delivery cost - which should be 0 for free shipping to the total, so total totals become different. Why this is happening? It looks like there is something wrong in Prestashop itself, not in the Laybuy module.

Even looking at the screenshot above one can notice something not quite right: there is a subtotal of £56.00, plus a discount of £6.95, plus free shipping, so the final total should be shown as £49.05. However, it is shown as £56.00. This is because the shown discount £6.95 as actually is a discount for free shipping - normal shipping cost is £6.95, so adding a discount of £6.95 gives customer a free shipping. But looking at the given pictures it is difficult to realize. (However, we didn't get any complaints from our customers.) I think the line "Discount(s) - £6.95" shouldn't be shown at all, or have some comment. Where is the error, in our Prestashop theme (highly customized), or in the Promotions and discounts module, which we use? Or in the core Prestashop code? I don't know.

Interesting, the situation with the Laybuy module is exactly the opposite. Laybuy gets order totals, using core Prestashop classes, and somehow gets £6.95 as a shipping cost. Which should be 0 as Free shipping discount applies to orders over £50. This discount is created using Promotions and discounts module.
Where is the error? Most probably in the same place as the first error with showing discount to the customer, that is in Prestashop itself. I have a look at the Prestashop code… well, it doesn't seem possible to find and fix. I can only make things worse. So instead I decided to "fix" the Laybuy module, adding a simple patch to its code. If totals are not equal - make them equal by subtracting the shipping cost. Easy-peasy!

Where is the error? Most probably in the same place as the first error with showing discount to customer, that is in Prestashop itself. I have a look at the Prestashop code... well, it isn't seems possible to find and fix. I can only make things worse. So instead I decided to "fix" the Laybuy module, adding a simple patch to its code. If totals are not equal - make them equal by subtract the shipping cost. Easy-peasy!

/**
    * Process the Order Items
    *
    * @return array
    * since 1.0.0
    */
    private function _processItems() {

        $tax = floatval($this->cart->getOrderTotal() - $this->cart->getOrderTotal(false));

        // items total
        $shipping_total = $this->cart->getOrderTotal(false, Cart::ONLY_SHIPPING);

        //vallka:
        $items_price = $this->cart->getOrderTotal(false, Cart::BOTH_WITHOUT_SHIPPING);
        $amount = $this->cart->getOrderTotal();

        if (Laybuy::$debug) {
            PrestaShopLogger::addLog("laybuy-vallka:$amount,$items_price,$tax,$shipping_total", 3, NULL, "Laybuy", 1);
        }

        if ($items_price+$tax == $amount) {
            $shipping_total = 0;
            PrestaShopLogger::addLog("laybuy-vallka-corrected!:$amount,$items_price,$tax,$shipping_total", 3, NULL, "Laybuy", 1);
        }
        //vallka end


        // shipping
        if ($shipping_total > 0) {
            $items[] = array(
                'id' => 'shipping_fee#' . $this->cart->id,
                'description' => 'Shipping fee',
                'quantity' => '1',
                'price' =>  $shipping_total
            );
        }

        // tax
        if ($tax) {
            $items[] = array(
                'id' => 'total_tax_amount_for_order#' . $this->cart->id,
                'description' => 'Tax amount for this order',
                'quantity' => '1',
                'price' => $tax
            );
        }

        $items[] = [
            'id'          => 'item_for_order___#' . $this->cart->id,
            'description' => 'Items',
            'quantity'    => 1,
            'price'       => $this->cart->getOrderTotal(false, Cart::BOTH_WITHOUT_SHIPPING)
        ];

        return $items;
    }

And it does work!

What Cart::BOTH_WITHOUT_SHIPPING parameter value for getOrderTotal method is, I can only guess… Well, I cannot even guess what meaning Prestashop developers wanted to give to it….

My fixed version of Laybuy module is here: https://github.com/vallka/prestashop-laybuy

I will make a pull request to the original repository to let them know. Maybe my fix isn't 100% correct. Or it can be done in a more simple way. I also hope the Prestashop developers will notice this problem and at least comment my post. But for a time being the fix is working and we can use Laybuy / Prestashop / Promotions and discounts module together without errors.

Prestashop Module — Enhanced Product List in Admin

Valentine — Wed, 05 May 2021 20:18:36 +0000

Today we are returning to Prestashop modules programming. What game shall we play today?

There are a number of modules which allows us to customize catalog view - in the front end. But what if we want to customize the back office? Namely, we are missing some columns in Product List view. It would be nice to see all our EAN13 codes in the list, as well as such values like Cost Price and Weight - these values are often set being set incorrectly by users and usually it is difficult to trace. Only when you trying to post an order which surprisingly weights 0 grams and costs 0 pence to ship you are starting to realize that some values are missing in the product description.

We have all the means to add columns. First of all, there is a hook actionAdminProductsListingFieldsModifier - and it does what the name suggests - it allows to modify fields in the select which is used to build the product list.

actionAdminProductsListingFieldsModifier
Located in: /src/Adapter/Product/AdminProductDataProvider.php
Parameters:

<?php
array(
  '_ps_version' => (string) PrestaShop version,
  'sql_select' => &(array),
  'sql_table' => &(array),
  'sql_where' => &(array),
  'sql_order' => &(array),
  'sql_limit' => &(string),
);

(It always helps to keep a copy of full Prestashop sources at hand. Looking at source file often helps to get the idea. In my case first came the discovery that this hook is called twice, most probably by mistake. Ok, mistake or not, now we are prepared to deal with this fact.)

Let's create a module skeleton, as we already did in the previous article. Go here:

https://validator.prestashop.com/generator

I couldn't find the needed hook in the dropdown, so we'll add it later.

After downloading and unzipping our brand new module, let's add code. Let's add the needed hook:

public function install()
{
        Configuration::updateValue('ADMINEXTRAINFO_LIVE_MODE', false);

        return parent::install() &&
            $this->registerHook('backOfficeHeader') &&
            $this->registerHook('actionAdminProductsListingFieldsModifier');  // added line
}

in the install method, and the hook implementation at the bottom of the file:

public function hookActionAdminProductsListingFieldsModifier($param)
{
        /* Place your code here. */
        print('***==hookActionAdminProductsListingFieldsModifier==***');
        var_dump($param);
}

just to make sure it works.

And it does. This is what we immediately see in the page source:

***==hookActionAdminProductsListingFieldsModifier==***array(10) {
  ["_ps_version"]=>
  string(7) "1.7.7.3"
  ["sql_select"]=>
  &array(16) {
    ["id_product"]=>
    array(3) {
      ["table"]=>
      string(1) "p"
      ["field"]=>
      string(10) "id_product"
      ["filtering"]=>
      string(4) " %s "
    }
    ["reference"]=>
    array(3) {
      ["table"]=>
      string(1) "p"
      ["field"]=>
      string(9) "reference"
      ["filtering"]=>
      string(13) "LIKE '%%%s%%'"
    }
    ["price"]=>
    array(3) {
      ["table"]=>
      string(2) "sa"
      ["field"]=>
      string(5) "price"
      ["filtering"]=>
      string(4) " %s "
    }
        ...

Scrolling down, we see that this is repeated twice - exactly what we expect from examining the code in AdminProductDataProvider.php. Ok, but the content is self-explaining. We only need to add new fields to the param in the same format. A very simple task, knowing that the fields we need are located in the same tables... And do not forget to prevent double execution.

This is what we will put in the hook:

public function hookActionAdminProductsListingFieldsModifier($param)
{
        /* Place your code here. */
        if (!$param['sql_select']['ean13']) {
                $param['sql_select']['ean13'] = array();
                $param['sql_select']['ean13']["table"] = "p";
                $param['sql_select']['ean13']["field"] = "ean13";
                $param['sql_select']['ean13']["filtering"] = "LIKE '%%%s%%'";
        }

        if (!$param['sql_select']['weight']) {
                $param['sql_select']['weight'] = array();
                $param['sql_select']['weight']["table"] = "p";
                $param['sql_select']['weight']["field"] = "weight";
                $param['sql_select']['weight']["filtering"] = ' %s ';
        }

        if (!$param['sql_select']['wholesale_price']) {
                $param['sql_select']['wholesale_price'] = array();
                $param['sql_select']['wholesale_price']["table"] = "sa";
                $param['sql_select']['wholesale_price']["field"] = "wholesale_price";
                $param['sql_select']['wholesale_price']["filtering"] = ' %s ';
        }
}

But obviously this is not enough. We need to display these new fields. Luckily for us, if we search Prestashop Module Development documentation, there is a page which refers exactly to our case! We need to add a subtree to module's views subfolder like this:

├───sql
├───translations
├───upgrade
└───views
    ├───css
    ├───img
    ├───js
    ├───PrestaShop
    │   └───Admin
    │       └───Product
    │           └───CatalogPage
    │               └───Lists
    └───templates
        └───admin

And the content of this subdir will be taken from this subfolder of Prestashop sources:

prestashop_1.7.7.3/src/PrestaShopBundle/Resources/views/Admin

Initially we will override only one file,

/src/PrestaShopBundle/Resources/views/Admin/Product/CatalogPage/Lists/list.html.twig

I never heard of .twig templates before, but what I see here looks exactly like Django templates or Jinja templates. Looks like all templating languages uses the same syntax... Good for us! (But need to notice that this syntax is slightly different from Smarty used in the front end and older parts of the back end.)

We see that products are displayed inside an HTML table and list.html.twig represents a tbody row. For simplicity, lets add EAN13 field in the same column as the product id, to keep thead unchanged:

 <td>
        <label class="form-check-label" for="bulk_action_selected_products-{{ product.id_product }}">
                {{ product.id_product }}
        </label>
        <br/>
        <a href="{{ product.url|default('') }}#tab-step6">{{ product.ean13 }}</a>
    </td>

I copied the idea from some other field. #tab-step6 in the link refers to the tab "options" in product description, where EAN13 is being set. The same way, let's put "weight" field under product category and wholesale price under price:

<td>
        {{ product.name_category|default('') }}
        <br>
        <a href="{{ product.url|default('') }}#tab-step4">{{ product.weight|round(2) }} kg</a>
</td>
<td class="text-center">
        <a href="{{ product.url|default('') }}#tab-step2">{{ product.price|default('N/A'|trans({}, 'Admin.Global')) }}</a>
        <br>
        <a href="{{ product.url|default('') }}#tab-step2">{{ product.wholesale_price|round(2) }}</a>
</td>

We need to use round(2) filter to give our numbers a good look, otherwise all 8 or so decimal digits will be displayed. Why it is not needed for product.price? Because it is done inside a php code, together with prepending (or appending) a currency sign. We would want to do the same with the wholesale price, but for simplicity let's live it as is for now. Let's also skip internationalization for 'kg' word.

To change the table header, we need to override, well, products_table.html.twig template. And what I have found, trying to implement filtering and sorting by our new fields - it doesn't work. Although we have defined, and hopefully correctly, all needed attributes for the new fields to be used in filtering and sorting, we will also need to modify some javascript somewhere. Filtering and sorting are performed by ajax call, and additional programming is required. We need to teach some javascript to deal with the new fields. Let's leave it for the next exercise. (And for my client's purpose this unfinished solution is already a big help.)

So below is my version of products_table.html.twig:

{% block product_catalog_form_table_header %}
    <tr class="column-headers">
        <th scope="col" style="width: 2rem"></th>
        <th scope="col" style="width: 6%">
            {{ ps.sortable_column_header("ID"|trans({}, 'Admin.Global'), 'id_product', orderBy, sortOrder) }}
        </th>
        <th scope="col">
            {{ "Image"|trans({}, 'Admin.Global') }}
        </th>
        <th scope="col">
            {{ ps.sortable_column_header("Name"|trans({}, 'Admin.Global'), 'name', orderBy, sortOrder) }}
        </th>
        <th scope="col" style="width: 9%">
            {{ ps.sortable_column_header("Reference"|trans({}, 'Admin.Global'), 'reference', orderBy, sortOrder) }}
        </th>
        <th scope="col">
            {{ ps.sortable_column_header("Category"|trans({}, 'Admin.Catalog.Feature'), 'name_category', orderBy, sortOrder) }}
        </th>
        <th scope="col" class="text-center" style="width: 6%">
            {{ ps.sortable_column_header("Price ex VAT"|trans({}, 'Admin.Catalog.Feature'), 'price', orderBy, sortOrder) }}
        </th>
        <th scope="col" class="text-center" style="width: 6%">
            {{ "Price VAT"|trans({}, 'Admin.Catalog.Feature') }}
        </th>

        {% if 'PS_STOCK_MANAGEMENT'|configuration %}
        <th scope="col" class="text-center" style="width: 6%">
            {{ ps.sortable_column_header("Quant"|trans({}, 'Admin.Catalog.Feature'), 'sav_quantity', orderBy, sortOrder) }}
        </th>
        {% else %}
            <th></th>
        {% endif %}

        <th scope="col" class="text-center">
            {{ ps.sortable_column_header("Status"|trans({}, 'Admin.Global'), 'active', orderBy, sortOrder) }}
        </th>
        {% if has_category_filter == true %}
            <th scope="col">
                {{ ps.sortable_column_header("Pos"|trans({}, 'Admin.Global'), 'position', orderBy, sortOrder) }}
            </th>
        {% endif %}
        <th scope="col" class="text-right" style="width: 3rem; padding-right: 2rem">
                {{ "Actions"|trans({}, 'Admin.Global') }}
        </th>
    </tr>
    <tr class="column-headers">
        <th></th>
        <th>
            {{ "EAN13"|trans({}, 'Admin.Catalog.Feature') }}
        </th>
        <th></th>
        <th></th>
        <th></th>
        <th>
            {{ "Weight"|trans({}, 'Admin.Global') }}
        </th>
        <th>
            {{ "Cost Price"|trans({}, 'Admin.Catalog.Feature') }}
        </th>
        <th></th>
        <th></th>
        <th></th>
        {% if has_category_filter == true %}
            <th></th>
        {% endif %}
        <th></th>
    </tr>
{% endblock %}

Here is a Github repository:

https://github.com/vallka/prestamodule_adminextrainfo

To be continued...

Pandas in the Pandaemic. (COVID-19 in Scotland — statistics) Part 3

Valentine — Wed, 14 Apr 2021 22:05:35 +0000

This is the 3rd part of an exploration of pandas package.

Continue reading on Medium »

UPD (2022-02-08):
The URL of myCovidash has changed to
https://www.vallka.com/mycovidash/
and it uses Django now instead of Flask. But still working fine for nearly a year without serious adjustments. And death rate is currently lower then ever indeed...

https://dailysceptic.org/2022/02/08/deaths-9-below-average-when-will-the-government-declare-the-pandemic-over/

The graph on the above link shows stats in England and Wales, mine is for Scotland. Similar figures... Looks like the end is near?

Pandas in the Pandaemic. (COVID-19 in — Scotland statistics) Part 2

Valentine — Wed, 31 Mar 2021 21:38:34 +0000

This is the 2nd part of the exploration of pandas package. Part 1 can be found here…

Continue reading on Medium »

Leaning pandas. Data manipulation and visualization. (COVID-19 in — Scotland statistics)

Valentine — Sun, 14 Mar 2021 20:26:16 +0000

Let’s do a little bit of pandas. Pandas is (or are? :) extremely popular. Let’s just dive in.

Continue reading on Medium »

Prestashop Modules Programming. Bcc outgoing emails

Valentine — Sun, 28 Feb 2021 19:50:40 +0000

What is the idea?

Let's start programming Prestashop modules. Where to start from? Here is the documentation:

https://devdocs.prestashop.com/1.7/modules/

Ok. But what would be our first module? It should be very simple, and yet it should be useful. Here is the idea: add BCC to all outgoing emails, that we will have copies of all outgoing emails in out mailbox. Why would we want it? Actually this is what many people want, just try to google. For newly set up website it may be useful. You may want to see how actually the emails look like. And find all versions of emails which Prestashop sends to your customers. Probably after looking to these emails you would want to change them a bit, maybe remove a default "Powered by Prestashop" line at the bottom... You may want to keep monitoring outgoing emails to ensure that next update of Prestashop do not revert your custom emails to default ones (you probably forgot to check "Keep email templates" while updating Prestashop). Ok, let's do it.

Among Prestashop hooks there is one which can be used for our purpose: actionEmailSendBefore

actionEmailSendBefore

Before sending an email This hook is used to filter the content 
or the metadata of an email before sending it or even prevent its sending

Located in: /classes/Mail.php

https://devdocs.prestashop.com/1.7/modules/concepts/hooks/list-of-hooks/

If we have a chance to look inside /classes/Mail.php file, we will see that all the parameters are prefixed with "&", what means we can change them:

$hookBeforeEmailResult = Hook::exec(
        'actionEmailSendBefore',
        [
            'idLang' => &$idLang,
            'template' => &$template,
            'subject' => &$subject,
            'templateVars' => &$templateVars,
            'to' => &$to,
            'toName' => &$toName,
            'from' => &$from,
            'fromName' => &$fromName,
            'fileAttachment' => &$fileAttachment,
            'mode_smtp' => &$mode_smtp,
            'templatePath' => &$templatePath,
            'die' => &$die,
            'idShop' => &$idShop,
            'bcc' => &$bcc,
            'replyTo' => &$replyTo,
        ],
        null,
        true
    );

I couldn't find any place in Prestashop Back Office which could potentially use BCC. So we can assume (for simplicity) it is always empty and we can just set it to our own value in the hook.

Let's create a module!

Luckily, there is a simple way to create a module skeleton. There is a module generator supplied by Prestashop itself:

https://validator.prestashop.com/generator

I wouldn't say it works perfectly but it works. Probably, it is not updated as ofter as new Prestashop versions are released, and contains some bugs. Also, as mentioned in the documentation, if you want to create a Payments module for Prestashop 1.7 you should not use this generator. But for our purpose it as just invaluable.

So put some values to start a module - name, version, author name etc.:

On next page let's select the the following - Yes to Confirm uninstall, No to Create default database table and Need instance. Compliancy min-max - 1.7.x (we do not bother about older versions).

Finally, we are asked about the hooks we want to use. Unfortunately, the generator seems not to know about the hook we are after:

Ok, let's just select a random one, we will change the code later:

Click on Create! button - the generator will generate bcc_outgoing_emails.zip file. This is how we named our module.

What is inside?

Inside the zip file there is a directory with the same name - bcc_outgoing_emails. And inside this directory:

Actually, we don't need most of these files. But it is interesting to look inside and see what is generated.

/bcc_outgoing_emails.php

This is the main ,php file. By default, Prestashop puts everything in this file. I would rather separate install/uninstall code, admin actions, hooks into different .php files. But for our very simple example the default behavior is good enough.

/sql/

We have actually selected No to Create default database table option, so this directory shouldn't be here at all. We don't need any sql content, so we can simply delete this directory.

/translations/

Ok, let's keeps it. We may want to use it to translate Back Office form. We won't do it now, but it good to know that we have this ability by default.

/upgrade/

Let's keep it as is.

/views/

Under /views/ directory Prestashop module keeps all templates, images, javascripts and css files. For both front end and back end. Definitely we don't need any front end files, and hardly we need any styling or images for back end. Or javascript. But it is interesting to see how all these files are going to be used in case they were needed.

Let's look inside the main file, bcc_outgoing_emails.php. First what we can notice - the generator correctly set author and version info in php class, but left

*  @author    PrestaShop SA <contact@prestashop.com>
*  @copyright 2007-2021 PrestaShop SA

in the top file comment. We may want to change the comment :)

Ok. More important things. We have chosen (randomly) actionAdminControllerSetMedia hook, but here we see 2 more hooks registered: header and backOfficeHeader:

return parent::install() &&
        $this->registerHook('header') &&
        $this->registerHook('backOfficeHeader') &&
        $this->registerHook('actionAdminControllerSetMedia');

(lines 66-69)

header hook is used for adding js and css files to front end:

 /**
 * Add the CSS & JavaScript files you want to be added on the FO.
 */
public function hookHeader()
{
    $this->context->controller->addJS($this->_path.'/views/js/front.js');
    $this->context->controller->addCSS($this->_path.'/views/css/front.css');
}

(lines 213-220)

We won't be touching front end at all, so we can delete all these lines together with files in /views/js/ and /css/ directories.

Similarly, backOfficeHeader hook is used to add back office js and css files:

/**
* Add the CSS & JavaScript files you want to be loaded in the BO.
*/
public function hookBackOfficeHeader()
{
    if (Tools::getValue('module_name') == $this->name) {
        $this->context->controller->addJS($this->_path.'views/js/back.js');
        $this->context->controller->addCSS($this->_path.'views/css/back.css');
    }
}

(lines 202-211)

Please notice that our back office js and css will only be added to the page with our module configuration form. Yes, it may have sense in many cases, but just be aware of it. In another my project I wanted back.js file to be always available in back office and had to spend some time trying to find out and get rid of the condition:

if (Tools::getValue('module_name') == $this->name) {

Another thing to notice here. These two hooks have been added automatically behind the scenes. Accidentally we have selected for our module a hook actionAdminControllerSetMedia. It looks like this particular hook is intended to be used instead of backOfficeHeader for adding js and css files. This is an open question to Prestashop developers - which hook should be actually used for this purpose and did they forgot to update the generator?

Anyway, both hooks seems to be working for this purpose (as I find out building another module), and we don't need any media in this project, so we can simply delete the lines :)

Let's finally write some code!

Let's delete all mentions of header and backOfficeHeader hooks and change actionAdminControllerSetMedia to what we really need: actionEmailSendBefore

public function install()
{
    Configuration::updateValue('BCC_OUTGOING_EMAILS_LIVE_MODE', false);

    return parent::install() &&
        $this->registerHook('actionEmailSendBefore');
}

... and below:

public function hookActionEmailSendBefore($param)
{
    /* Place your code here. */
}

We know that all hooks get one and only argument - an array of various parameters which differ from hook to hook. It seems like a bug that generator didn't include this $param array in the skeleton, so we have to add it ourselves.

As we are going to write some real code here, it's time to look for our own configuration. Where will we take to email address to which we going bcc? And very handy, the generator did generate some basic configuration for us, and luckily, this is almost exactly what we need! The generator has created 3 configuration variables:

'BCC_OUTGOING_EMAILS_LIVE_MODE' => Configuration::get('BCC_OUTGOING_EMAILS_LIVE_MODE', true),
'BCC_OUTGOING_EMAILS_ACCOUNT_EMAIL' => Configuration::get('BCC_OUTGOING_EMAILS_ACCOUNT_EMAIL', 'contact@prestashop.com'),
'BCC_OUTGOING_EMAILS_ACCOUNT_PASSWORD' => Configuration::get('BCC_OUTGOING_EMAILS_ACCOUNT_PASSWORD', null),

This is almost exactly what we need! We don't need any password, so lets delete all lines where BCC_OUTGOING_EMAILS_ACCOUNT_PASSWORD is mentioned, and for clarity change the name BCC_OUTGOING_EMAILS_ACCOUNT_EMAIL to BCC_OUTGOING_EMAILS_BCC_EMAIL. Let's even keep BCC_OUTGOING_EMAILS_LIVE_MODE variable - using it will add some professional look to our module :)

Here is how the hook handle will look like:

public function hookActionEmailSendBefore($param)
{
    if (Configuration::get('BCC_OUTGOING_EMAILS_LIVE_MODE', false)) {
        $bcc = Configuration::get('BCC_OUTGOING_EMAILS_BCC_EMAIL', '');
        if ($bcc) {
            $param['bcc'] = $param['bcc']? "{$param['bcc']},$bcc": $bcc;
        }
    }
}

UPD. And here we have an interesting result. I assumed that multiple email addresses in bcc field should be separated by comma. But it appeared to be wrong. In reality it would be difficult to get to to the point where this problem would show. But working on this project, I had 2 copies of the same module installed (by different names) and Prestashop began to complain about wrong format of bcc parameter. It occurred that Prestashop Mail.php expects multiple email addresses in to and bcc fields as arrays. And another interesting result from investigating Mail.php - there is no possibility to set cc at all. Well, I may understand the logic, but is looks at least surprising for me...

So the final version of the hook handler is:

public function hookActionEmailSendBefore($param)
{
    if (Configuration::get('BCC_OUTGOING_EMAILS_LIVE_MODE', false)) {
        $bcc = Configuration::get('BCC_OUTGOING_EMAILS_BCC_EMAIL', '');
        if ($bcc) {
            if (! $param['bcc']) {
                $param['bcc'] = $bcc;
            }
            else {
                if (is_array($param['bcc'])) {
                    array_push($param['bcc'],$bcc);
                }
                else {
                    $param['bcc'] = [$param['bcc'],$bcc];
                }
            }
        }
    }
}

Well, a bit more complicated than it was expected :)

Anyway, the whole investigation had rather theoretical value. In practice, I think, the simplest solution to overwrite old bcc setting with a new one would satisfy 99.9% cases.

Done and tested.

Other thing to notice: while the generator generated three configuration variables (we got rid of one of those), in install and uninstall methods only one is mentioned. Looks like the other two were just forgotten, at least in unistall method. Let's add our second variable to uninstall:

public function uninstall()
{
    Configuration::deleteByName('BCC_OUTGOING_EMAILS_LIVE_MODE');
    Configuration::deleteByName('BCC_OUTGOING_EMAILS_BCC_EMAIL');

    return parent::uninstall();
}

Let's leave install method as it is. I think the initialization of BCC_OUTGOING_EMAILS_LIVE_MODE is not needed altogether (and I have changed the default for this ver from true to false - seems more logical for me). but let's leave it alone.

We didn't look at the /views/templates/admin/configure.tpl file, but it basically OK for our purpose. We may want to delete the example texts on the page, update links to the documentation when we had one, but this is straightforward. Just textual changes.

Deploy and install

First we need to create (or rather re-create) the zip file. After deleting the files and directories we didn't use, here is the structure:

Do not forget to put all these file in bcc_outgoing_emails folder, as they were in the original zip. Then go to back office in Prestashop, Modules->Module Manager->Upload a Module. Configuration is self-explaining. Voilà! Our first Prestashop module is installed and working.

Yet another potential problem.

When trying to test a newly installed module, first thing I went to Prestashop Email configuration->Test your email configuration. I expected a test email to be bcc'd too, accordingly the module setting. Nope. It just doesn't work with Test your email configuration. So you need, for example, to register a new user to see any effect. The Welcome email will be sent to bcc address.

GitHub repositary

https://github.com/vallka/bcc_outgoing_emails

How to deploy Django application to HostPresto!

Valentine — Thu, 18 Feb 2021 16:38:59 +0000

In this tutorial we will deploy Django Polls project from official Django tutorial to the HostPresto!

Continue reading on Medium »

How to start a Django application from scratch on HostPresto!

Valentine — Mon, 15 Feb 2021 17:02:13 +0000

HostPresto!

In this tutorial we will create a brand new Django application on HostPresto!

Let’s start by setting up our environment:

Creating a subdomain:

Suppose you have a domain registered with HostPresto! — yourname.xyz. For this tutorial, we will create a subdomain: start.youname.xyz, as you may already have an existing website sitting on the main domain. Adding a subdirectory like youname.xyz/start is less convenient, in our opinion.

Log in into your cPanel and click on Subdomains in the DOMAINS section. Enter ‘start’ into Subdomain field. Select your main domain (youname.xyz) from Domain dropdown. The Document Root field will be auto-populated with ‘start.yourname.xyz’. Let’s accept the default value. Click on Create.

The Subdomain has now been created. If you point your browser to http://start.yourname.xyz, you should normally see an empty directory listing — something like this:

Setting up a Python application:

In cPanel you will then need to go to the SOFTWARE section and select Setup Python App. Then click on CREATE APPLICATION.

HostPresto! Uses software from Passenger ( https://www.phusionpassenger.com/) to set up Python applications. The setup page isn’t very intuitive, in my opinion. But let’s use it anyway.

Select the latest Python version in the first field. It is 3.7.3 at the time of writing.

In Application root type in the directory name we have created in the previous step, relating to your home directory — that is, just ‘start.yourname.xyz’.

Select ‘start.yourname.xyz’ as the Application URL in the next line.

Leave Application startup file and Application Entry point empty — the system will create this file by its own rules and let it do it.

The last thing — it’s sensible to set up a log file (Passenger log file) — enter /home/yournamexy/start.log in the Passenger Log File box, replacing “yournamexy” with your cPanel username.

Click on CREATE

Click on START APP

Again, point your browser to http://start.yourname.xyz (or refresh the page if you still have it in one of the tabs). Now it should show something like this:

This means we have successfully created a virtual environment for our Python application.

Installing Python packages:

Yes, we can do it directly from the web interface (by going to the ‘requirements.txt’ file — if you have this file in root subdirectory, In the ‘Configuration files’ section, click ADD, then click ‘Run Pip install’), but we won’t do it this way. Let’s do it the traditional way, via running shell commands. We will need to use shell commands anyway, so let’s go to the command line straight away.

We need to make a ssh connection to our server. If you are on Windows, I recommend to use a program called PuTTY. Use your cPanel username and password (not client area password!) and change port 22 to 88 for the ssh connection.

Once logged in, activate your project virtual environment and cd into the project directory. How do you do this? Actually cPanel contains a useful hint: at the top of the page the command is displayed for you:

Enter to the virtual environment. To enter to virtual environment, run the command: source /home/yournamexy/virtualenv/start.yourname.xyz/3.7/bin/activate && cd /home/yournamexy/start.yourname.xyz

Just click on the command shown on your cPanel and it will be copied to your clipboard.

Paste the command to PuTTY terminal window and run it — now you are in your project directory and the virtual environment has been activated. First of all, let’s upgrade pip — run the following command:

pip install --upgrade pip

Next, we will install Django:

pip install django

We might need mysqlclient, so let’s install it:

pip install mysqlclient

I highly recommend to install WhiteNoise for managing static files, this will make life much easier:

pip install whitenoise

To finalise set up, let’s create the requirements.txt file:

pip freeze > requirements.txt

This command will create the requirements.txt file, which can be used to re-create the environment if needed. We don’t need this file right now right here, but it’s a good practice to have it at hand just in case.

Creating a project:

Run the following command:

django-admin startproject start

(Please notice the dot at the end of command. This dot tells django-admin to create a project in the current directory. Otherwise, it will create a subdirectory ‘start’ in the current directory — this is not what we want.)

Now let’s try to create an app:

python manage.py startapp startapp

Most probably, you will get a long list of errors when running the above command. The very last line contains the actual error, something like this:

django.core.exceptions.ImproperlyConfigured: SQLite 3.8.3 or later is required (found 3.7.17).

Actually, it’s good that we’ve run into this error in the very first step. (If not and your command completed successfully you still need to read further!).

As you may remember, for local development you may have used the SQlite server, built-in Python SQL server. This is completely fine for local development, but totally unacceptable for a production server (and we are pretending to create a production server now). So luckily for us, Django doesn’t like the version of the installed SQlite server, so there is no other way than to switch to MySQL. (Traditionally Django uses PostgreSQL but in our case we have MySQL pre-installed on the server, so we will use MySQL)

So let’s return to cPanel and create a MySQL database, using the available cPanel tools.

Creating MySQL database for the project:

This is easy. In cPanel, go to DATABASES — MySQL Database Wizard.

Type in a name for a database. Let’s call it ‘start’ in order to be consistent.

cPanel automatically prefixes the database name with yournamexy_, so the full name will be ‘yournamexy_start’.

Click on Next Step , provide a name and a password for the new database user (let’s give the user the same name: ‘yournamexy_start’). Don’t forget to make a note of the newly created password — we’ll need it in a minute.

Click Next.

On Step 3 check the ALL PRIVILEGES tick box.

Click Next.

Configuring Django to use MySQL:

In the cPanel file manager, navigate to /home/yournamexy/start.yourname.xyz/start. This subdirectory should contain the settings.py file. Open this file in the file editor and find the DATABASES section. Replace the whole section with the following lines:

DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'yournamexy_start', 'USER': 'yournamexy_start', 'PASSWORD': os.environ['START_DB_PASSWORD'], 'HOST': 'localhost', } }

(Replace yournamexy_start with your own name you have created in the previous step)

It’s not a good idea to hard code the database password, so we will read it from the environment variable instead.

So, return to the SSH console window you have open and set a variable for a password (Replace “your password” in the below command with the password you have chosen when creating a database):

export START_DB_PASSWORD=[your password]

Now you should be able to create an application. Run the following command again:

python manage.py startapp startapp

(Now I realise that I have chosen a bad name for my app — ‘startapp’. startapp is also a command in manage.py, so it is unclear what is what in the above command. It would have been better to name the app, say, ‘first_app’, and the command to create this app would then be: ‘ python manage.py startapp first_app')

Configuring passenger_wsgi.py:

Return to your cPanel — Setup Python App page.

You will see the Application startup file and Application Entry point fields are populated now.

The Application startup file is called passenger_wsgi.py. Find this file in the root folder of our project. Open it in File Manager. It should look like this:

import os import sys

sys.path.insert(0, os.path.dirname( __file__ ))

def application(environ, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) message = 'It works!\n' version = 'Python %s\n' % sys.version.split()[0] response = '\n'.join([message, version]) return [response.encode()]

We don’t need all of this, so delete the content and replace it with the following:

import os from django.core.wsgi import get_wsgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'start.settings') application = get_wsgi_application()

The last thing that we need to do before attempting to run our new Django site, is to set an environment variable for the db password in the Python web application. We did this previously via the SSH command shell, but we now need to do the same for the webserver.

For this, return to the SOFTWARE section in cPanel, go to Setup Python App , click on the Edit (pencil) icon, in line with our newly created application start.yourname.xyz.

In the Environment variables section add a variable named START_DB_PASSWORD with your database password. Click SAVE at the top of the page.

Click RESTART.

Now we are ready to try the application. Point your browser to http://start.yourname.xyz and refresh the page. Most likely you will see a message such as:

DisallowedHost at /Invalid HTTP_HOST header: ‘start.yourname.xyz’. You may need to add ‘start.yourname.xyz’ to ALLOWED_HOSTS. ALLOWED_HOSTS = []

Thankfully, the message is very clear. ALLOWED_HOSTS = [] in settings.py. This default value only allows us to run a web app on the localhost. Now we need to change this. Open your settings.py file and find the line:

ALLOWED_HOSTS = ['polls.yourname.xyz']

Reolace it with the following, Replacing ‘yourname.xyz’ with your actual subdomain:

Then RESTART the application again.

Refresh the http://start.yourname.xyz page in your web browser, and you should then see:

Congratulations! Our first Django application is up and running!

Further steps:

OK, Let’s continue forward. Let’s try the admin page: http://start.yourname.xyz

It looks like our web application isn’t currently seeing the .css files, causing the formatting to look broken.

Luckily, we already installed the WhiteNoise package, which will help us with just one line in settings.py. Open your settings.py file again in the cPanel file editor and add the below to the MIDDLEWARE section:

MIDDLEWARE = ['django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', #new 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware',]

RESTART the app once more and then refresh the page in your web browser:

Much better!

This is what WhiteNoise does for us — no need to run the collectstatic command. All static files are now being served directly from the Python server.

python manage.py migrate

The next thing we need to migrate is the database. We didn’t write a single line of code yet, but Django did a lot for us. So let’s migrate by running the below command in our SSH terminal:

python manage.py createsuperuser

Next, let’s create a superuser:

Now you can log in to admin page via your web browser.

Creating a meaningful app:

OK, let’s create something meaningful. In this example lets create a simple Contact list.

First of all, we need to create a model. Let’s open a nearly empty file in the cPanel file manager: startapp/models.py and add some content to it. A very simple Contact model:

># Create your models here. class Contact(models.Model): forename = models.CharField(max_length=100) surname = models.CharField(max_length=100) email = models.EmailField(max_length=100)

def __str__ (self): return f"{self.forename} {self.surname}: {self.email}"

We also need to update our settings.py file one more time. We need to add our app, startapp, to the INSTALLED_APP section:

INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles',

'startapp', #new ]

Let’s add our new model to the admin area. Add these lines to the startapp/admin.py file:

from django.contrib import admin

# Register your models here. from .models import Contact admin.site.register(Contact)

python manage.py makemigrations

In the SSH console, run following command:

python manage.py migrate

Then:

Next, RESTART the app and then login to the admin interface in your web browser, using credentials you supplied to the createsuperuser command

In this tutorial, I won’t go further into creating views, forms and other common stuff. All this is common to any hosting provider, so you will find tons of useful tutorials on the Internet.

Instead, I’d like to pay some attention to details, specific for HostPresto. Namely, testing.

Testing our app:

Let’s create a very simple test in our startapp/tests.py file:

from django.test import TestCase

# Create your tests here. from .models import Contact

class ContactModelTests(TestCase): def test_contact_save(self): contact = Contact() contact.save()

self.assertNotEqual(contact.id, None,'id is autocreated') self.assertEqual(len(Contact.objects.all()),1,'Exactly 1 record in db after save')

python manage.py test

If we try to run test the usual way:

Got an error creating the test database: (1044, “Access denied for user ‘yournamexy_start’@’localhost’ to database ‘test_yournamexy_start’”)

Most likely, we will get an error, something like this:

This is because Django tries to create a new database for testing, and our database user does not have enough rights to create another database on the MySQL server.

OK, we can create an empty database via cPanel — MySQL Databases, and then use the parameter — keepdb, but there is another problem: cPanel requires that the database name starts with ‘yournamexy_’. We cannot prepend ‘test_’ to the name.

OK, there are still possibilities. Let’s create a database with the name ‘ yournamexy_start_test ’. This is allowed. How should we tell Django to use this name? We need to edit the startapp/settings.py file one more time and add an option to the DATABASES section:

DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'yournamexy_start', 'USER': 'yournamexy_start', 'PASSWORD': os.environ['START_DB_PASSWORD'], 'HOST': 'localhost',

'TEST': { 'NAME': 'yournamexy_start_test', } } }

python manage.py test --keepdb_Note: Don't even think of using the live db for testing - running tests on live will empty all data!_

Now we can run tests with the following command:

Logging:

Setting up logging in Django is really complicated. I won’t go in to detail here, but what can we do with the default configuration?

Actually, it just works, although to use it is not very convenient.

To demonstrate logging, let’s add some logging to our model via the startapp/models.py file:

from django.db import models import logging logger = logging.getLogger( __name__ )

# Create your models here. class Contact(models.Model): forename = models.CharField(max_length=100) surname = models.CharField(max_length=100) email = models.EmailField(max_length=100)

def __str__ (self): return f"{self.forename} {self.surname}: {self.email}"

def save(self, *args, **kwargs): super().save(*args,**kwargs) # Call the "real" save() method. logger.error( f"Contact saved with id={self.id}.")

The default logging level is set to ERROR (despite whatever is written in Django documentation — probably Passenger or cPanel somehow interferes with it), so don’t bother using logger.debug of logger.info.

Now go to your apps admin interface and create a few Contacts to see how it works.

Where can we see the logs?

Remember, there was a Passenger log file, of which the name we set up in the SOFTWARE — Setup Python App panel? (in this example we called it /home/yournamexy/start.log).

All server logs from our Python application go into this file, and logs generated by calling logger.error or logger.critical will also go into this file. Not very convenient, but at least we can use the default Python logging mechanism.

Another useful option would be to install Django Debug Toolbar , although it still requires some configuration for optimal use.

Conclusion:

In this tutorial, we have created a very basic Django application. This application does very little, and does not even have a proper front end — all it has is an admin area.

But we covered very important details, related to the specifics of How to set up a Python environment within cPanel on HostPresto!, how to switch to MySQL database usage, how to set up a passenger_wsgi.py file, how to use logging with minimum configuration effort and even how to test our application.

In most cases, using a remote server as a development server is not a very good idea. It would be better to install Python and all development tools locally and only use the remote server as a test or production environment.

All files for this tutorial can be found here:

vallka/django-start

HostPresto! is reasonably priced and reliable hosting. What makes it outstanding — like many hosting providers they give you PHP and MySQL out of the box, but unlike most of other providers, they give you a very simple solution to host almost any kind of application, Node.js, Django, Flask, Ruby-on-Rails to mention a few. And all this zoo of applications can co-exist on the same server! What makes HostPresto! a great choice for any developer.

HostPresto!

Originally published at https://hostpresto.com.