Using a domain stored in a field for a Many2one

Guewen Baconnier — Sat, 15 Oct 2022 18:00:47 +0000

It is a very common pattern in Odoo to filter available records in a Many2one depending on another field.

For instance, users can select a Stock Location only if it abides by some rules. You may have a computed field, that gives the allowed locations for a move line.

The issue

Let's take a fictive use case where we allow moving a product in a location only if this location is associated to this product.

In Python:




class StockMoveLine(models.Model):
    _inherit = "stock.move.line"

    allowed_location_ids = fields.Many2many(
        comodel_name="stock.location",
        compute="_compute_allowed_location_ids"
    )

    @api.depends("product_id", "move_id.location_dest_id")
    def _compute_allowed_location_ids(self):
        for line in self:
            destination = line.move_id.location_dest_id
            line.allowed_location_ids = destination.allowed_sublocations(line.product_id)

And in the XML view:



<field name="allowed_location_ids" invisible="1" />
<field name="location_dest_id" domain="[('id', 'in', allowed_location_ids)]"/>

It works quite well, until you have a large number of locations, e.g. 5000 allowed locations. When you change the product, an onchange is triggered to get the new allowed_location_ids, which is done very fast server_side, but the front-end starts to freeze (5-10 seconds).

This issue happens because we return 5000 ids to a Many2many widget, which has to process every single id.

ℹ️ I encountered this issue in Odoo 13.0, it may or may not still happen on newer versions.

The solution

Here comes to the rescue the OCA module web_domain_field.

As stated in its description:

In order to mitigate these limitations this new addon allows you to use the value of a field as domain of another field in the xml definition of your view.

Go read the description of the module as it gives other interesting use cases.

Using a domain set directly in the field short-circuits the Many2many widget on the view and fixes the performance issue.

Our code would now look like:

In Python:




class StockMoveLine(models.Model):
    _inherit = "stock.move.line"

    allowed_location_domain = fields.Char(
        compute="_compute_allowed_location_domain"
    )

    @api.depends("product_id", "move_id.location_dest_id")
    def _compute_allowed_location_domain(self):
        for line in self:
            destination = line.move_id.location_dest_id
            allowed_locations = destination.allowed_sublocations(line.product_id)
            line.allowed_location_domain = json.dumps([("id", "in", allowed_locations.ids)])

And in the XML view:



<field name="allowed_location_domain" invisible="1" />
<field name="location_dest_id" domain="allowed_location_domain"/>

The real use case used to showcase this Odoo module was in stock_storage_type.

Introduction to Odoo Components

Guewen Baconnier — Sat, 19 Jun 2021 13:04:15 +0000

One of the greatest strength of Odoo, besides its active community, is the extensibility provided through modules.

At the core of this extensibility is the model inheritance, with a concept of re-openable models.

Models extensions

A simplified definition of the partner model:

from odoo import models

class ResPartner(models.Model)
    _name = "res.partner"

    name = fields.Char(required=True)
    # ... other fields

    def _address_fields(self):
        """Returns the list of address fields that are synced from the parent."""
        return list(ADDRESS_FIELDS)

Any module can extend the model, add fields or methods. Here, the module partner_address_street3 which adds a third street field:

class ResPartner(models.Model):
    _inherit = "res.partner"

    street3 = fields.Char('Street 3')

    def _address_fields(self):
        fields = super()._address_fields()[:]
        fields.append('street3')
        return fields

If you are familiar with Odoo development, you probably already know this. If this is new to you and are interested in more details, you may start from the beginning.

Behind the scene
The magic behind this inheritance is that when Odoo starts, it finds all the sub-classes of models.Model (actually models.BaseModel) and builds a graph following the dependencies of the modules - defined in __manifest__.py in each module - and using the name of the model in _inherit attributes. From there, it dynamically creates an entirely new aggregated class with a MRO generated from its graph of model classes. This final class is what being used for actual instances of models.

The extensions possibilities are powerful, but they also have a weakness, encouraged by the framework, to grow what some would call fat models. Every module adds their own methods and logic in the models, which can lead to issues like namespace collisions, bad readability due to mixing of responsibities or duplicated logic.

I like the single responsibility principle and the idea of composition over inheritance when it makes sense. Well decoupled classes are easier to test as well.

That being said, we can say that what I am looking for is a way to define "Service classes", classes which can do one thing and do it well. Using a bare Python class would not be a solution, as it would not be extensible by other Odoo modules. But... Odoo has this already.

It is not widely used for this purpose, but the class models.AbstractModel can be described as a model which is not backed by a database table, therefore, it has no fields, only pure Python logic. Its name can be confusing, because it is not "abstract" in the pure OOP definition of a class that cannot be instantiated. It really is about the database storage.

The two main purposes I have seen for Abstract Models are:

"Real" abstract models which are then included in normal Models to share and provide additional fields or logic (mixin / multiple inheritance's way)
Services, which is related the topic of this article

Here is how a service based on an Abstract Model could be defined:

class SettlePaymentService(models.AbstractModel):
    _name = "settle.payment.service"

    def settle(self, payment):
        client = self._psp_client()
        client.settle(payment.id)

    # ...

And a model could then use this service with:

class PaymentTransaction(models.Model):
    _inherit = "payment.transaction"

    def button_settle(self):
        self.env["settle.payment.service"].settle(self)

If I'm speaking about services, this is because this is what Components are about. Are Abstract Models enough for this use case? Yes, they are. No, they aren't. "It depends." is often a pretty good answer to a technical question. I'll describe what they can do, and let you judge if you need them or not.

Let's begin with components

A bit of context

The Components are provided by an Odoo Community Association module. You can find it on GitHub

OCA / connector

Odoo generic connector framework (jobs queue, asynchronous tasks, channels)

🔥 It is not only for developing connectors, it is hosted in this project because it was born from this use case, and is a dependency for the Connector module.

It has a long history, as I started working on the first version of Connector in 2013, but it had a very different form by then. I wrote the Component module and reimplemented the Connector module on top of it in 2017, which means it was available since Odoo 10.0.

Some of the modules implemented on Components on top of my head are various connectors for different services (Jira, Magento, ...) the Odoo REST framework, the EDI project for Odoo, the storage suite, the barcode scanner application of the OCA WMS project... Let me know in the comments about the uses cases you know!

How to define a component?

The definition of a Component is much like a model:

from odoo.addons.component.core import Component

class SettlePayment(Component)
    _name = "settle.payment.service"

    def settle(self, payment):
        client = self._psp_client()
        client.settle(payment.id)

And the inheritance works the same way across modules:

class SettlePayment(Component)
    _inherit = "settle.payment.service"

    def settle(self, payment):
        _logger.info("added a logger when we settle")
        super().settle(payment)

You will have noted that until there, it looks pretty similar to the examples with Models above, with reason. The classes and inheritance mechanism are the same but:

The Component classes never have any database table.
There is no _inherits as it only makes sense for models with tables. _inherit works the same way.
There is no equivalent to TransientModel, it would not make any sense without table.
There is an AbstractComponent class, which can not be instanciated, it is only used as a "base interface / base implementation" for other Components to depend on.

You can get a component by its name from a Model, a bit like when you get a Model from self.env:

class PaymentTransaction(models.Model):
    _inherit = "payment.transaction"

    def button_settle(self):
        with self.env["payment.collection"].work_on(self._name) as work:
            settle_service = work.component_by_name("settle.payment.service")
            settle_service.settle(self)

But this is not really how it is meant to be used!
Also, I promise I'll get back later to what this work_on() is about.

Component matching by usage

This is where we really start, hang on!

Using models.AbstractModel for services is nice until you need more dynamic things.

For instance: if the type of the payment is "bank", use service BankExportService, if the type is "creditcard", use SettlePaymentService. Or, the export of a "res.partner" must be done using a CSV to a sFTP service, and the export of a "product.product" using a REST API.

This can be totally fine with some conditions in the model that would call one of the service. Or you can use the dynamic properties of Components. In the first case, the caller is responsible to choose the service, with components, the service is selected at runtime depending on the context without intervention on the caller's side.

Each Component gets assigned a _usage. Real life usages are "record.exporter", "record.importer", "search", "record.validator", "webservice.request", ... This is what will be used to find the component you need for the action you want to run.

With this new learning, let's write a Component again:

class RecordExporter(Component):
    _name = "generic.record.exporter"
    _usage = "record.exporter"

    def export(self, record):
        _logger.info("the generic exporter only logs data: %s", record.read())

The caller of the component:

class ResUsers(models.Model):
    _inherit = "res.users"

    def button_export(self):
        self.ensure_one()
        with self.env["my.collection"].work_on(self._name) as work:
            work.component(usage="record.exporter").export(self)

Why is it better to find my component by usage rather than by name? Because the name is fixed and is what is used to define the dependencies. If a module that I don't control defines base components and they are get by name in the middle of a chunk of code, I'm stuck. If they are get by usage, I can use the inheritance (_inherit) to modify the usage of the initial component, and create an entirely new or variant of the component with the expected usage (_usage). This is also what allows dynamic dispatching of components, as we will see in the next section.

In less words: the name defines the inheritance, the usage defines which component is used at runtime.

Want more?
Here we retrieve a component by usage with work.component(). When you are inside a component, you can use self.component().

This is the method you will use the most. It assumes that only a single component for the same usage exists for a context, otherwise it would not know which one to use.

In some cases, you may want to retrieve all the components for a usage and can use work.many_components()

Apply on... models?

As I was speaking about dynamic dispatch (maybe a language abuse, but I have no better words), here is an example.

On the components, the _apply_on attribute defines on which models a component may be used. The default value is None, which means all models.

I keep using the generic exporter defined above, but I will add a different exporter for products:

class CSVRecordExporter(Component):
    _name = "csv.record.exporter"
    _inherit = "generic.record.exporter"
    _usage = "record.exporter"

    _apply_on = ["res.partner"]

    def export(self, record):
        csv_data = self._generate_csv(record)
        self._export_to_sftp(csv_data)

The export method here builds a CSV and pushes it to a sFTP. The _inherit is not strictly mandatory, but often makes sense. The real addition for the example is _apply_on.

And the caller of the component... didn't change:

class ResPartner(models.Model):
    _inherit = "res.partner"

    def button_export(self):
        self.ensure_one()
        with self.env["my.collection"].work_on(self._name) as work:
            work.component(usage="record.exporter").export(self)

The code is 100% the same as the code we had for Users right? And yet, when we call button_export, the data will only be logged for users, and be exported as CSV for partners.

The _name of the 2 exporters is different, but we didn't had to change the caller of the service. Hopefully, you should start to see the potential for code sharing and extensibility (in a real example, button_export could then be added in an abstract model and included both in the users and the partner models, then the same method returns the appropriate component for each model).

Want more?
In a component, you can declare the _component_match method to define very finely which component to pick.

The following component can be used only if, for the current context (WorkContext, provided by work_on()) matches with the condition.

class BankExportService(Component):
    _name = "bank.exporter"
    _usage = "payment.exporter"

    @classmethod
    def _component_match(cls, work):
        return work.collection.payment_type == 'bank'

We can give additional context to the "WorkContext":

class PaymentTransaction(models.Model):
    _inherit = "payment.transaction"

    def button_export(self):
        self.ensure_one()
        with self.env["my.collection"].work_on(self._name, payment_type=self.type) as work:
            work.component(usage="payment.exporter").export(self)

Collection and `work_on()`

I promised I would get back to this work_on we saw in the earlier code fragments.

To explain this, let's start from the reason it has to exist. As the Components are very polyvalent, used for many different use cases, by different classes of modules, a problem I had to avoid was namespace collisions. What if I install a module A that defines a record.exporter, and module B does the same? We could have manually added prefixes in the usages, but it feels clunky.

An additional attribute exists on Components... the Collection.

Every component is assigned to a collection. Generally the base module for an implementation of a connector, of a generic framework such as REST or EDI... A collection could be "magento.backend" or "edi.backend".

In other terms, each collection has its own set of components. A collection is a Registry of components.
The Components _names are global, but for a given usage, components will eventually be filtered on the current collection.

A component with the collection set:

class RecordExporter(Component):
    _name = "record.exporter"
    _collection = "foo.collection"

🌈 you will pretty often see the words collection and backend used interchangeably, because historically, the Connector module was using solely the term "Backend". The latter did not really made sense for the more generic Component system, so it became Collection.

What then from this collection? It relates to a Model, which will be the reference for this collection. It can be a Model or an AbstractModel depending of the need. The benefit of a Model is that it can store fields, such as the URL of the service, the credentials, ...

class FooCollection(models.Model):
    _name = 'foo.collection'
    _inherit = 'collection.base'

The _name of the model has to match the _collection of its components, and the model has to _inherit from collection.base.

Now to the work_on context manager. It is made available by collection.base. It initializes and returns a WorkContext, which is the container for the current context (collection, model, env and arbitrary values) that makes the glue between the current "Context" (not the Odoo context, but the collection, model and values we want to use in components).

✨ The WorkContext is a bit like the env of Odoo but for Components.

work_on() bridges the gap between Odoo Models and Components. From a model, this is needed to be able to reach components.

class FooCollection(models.Model):
    _name = 'foo.collection'
    _inherit = 'collection.base'

    def export_all(self, records):
        with self.work_on(records._name) as work:
            work.component(usage="export.batch").export(records)

Or from another model:

class ResPartner(models.Model):
    _inherit = 'res.partner'

    def sync_from_foo(self):
        with self.env["foo.collection"].work_on("res.partner") as work:
            work.component(usage="sync").run_sync(self)

Once in a component, work_on() is not needed anymore, as every component holds the WorkContext and is able to reach other components.

Here is a simplified example that would use other components to export a record:

class FooSync(Component):
    _name = "foo.sync"
    _collection = "foo.collection"
    _usage = "sync"

    def run_sync(self, record):
        api_client = self.component(usage="webservice")
        data = api_client.read(record.id)
        odoo_data = self.component(usage="mapper").map(data)
        record.write(odoo_data)

This minimal component shows that we can split the work in small services, with their own responsibilities. Here, we have one service reading data from a webservice, and a second service which maps the external data to a dictionary that we can feed to write().

Want more?
Worth to note, the _name and _inherit attributes work the same way as Odoo Models, so for these, using prefixes is required. This is less an issue as the name is supposed to be used only for defining inheritance.

When working on an implementation using Components, I advise to define a MyCollectionBaseComponent(AbstractComponent) that sets the _collection, then to make all the components inherit from it: you won't need to set the _collection everywhere, and get the nice benefit of being able to customize all components of your implementation at once if needed (much alike the BaseModel).

When you call work_on(), you can pass arbitrary arguments, for instance, if you have several components that needs to use an API client transversaly, you can open the connection once, pass it to the WorkContext, and use it from the components.

            def export_all(self, records):
                api_client = self._client()
                api_client.connect()
                with self.work_on(records._name, api_client=api_client) as work:
                    work.component(usage="export.batch").export(records)

From the component:

            def export(self, records):
                api_client = self.work.api_client
                api_client.xxx

You actually can create a Component without collection, which can then be retrieved from any context. But this is really to use with caution, as it pollutes the global namespace. A good practice if you want to share components in several implementations is to create an AbstractComponent without collection, and to implement the component in each collection. This way, you can share code but keep more control on when/how it can be used.

The components are found by default according to the model passed to work_on(), but the work.component() method also accepts a model_name parameter, which will find the component for another model!

Summary

Components are used to build service classes. They are registered in Collections (registries of components). They have an inheritance system similar as the Model classes.

To be able to work with components, we use the context manager work_on() on the Collection. When we ask a component for a given usage, the collection's registry will match the components in this order:

Find a matching Component for the given usage (_usage) for the current collection (_collection) and model (_apply_on)
Find a matching Component for the given usage (_usage) for the current collection (_collection) and any model (no _apply_on on component)
Find a matching Component for the given usage (_usage) for any collection (no _collection on component) and any model (no _apply_on on component)

In any of the steps above, in case of multiple candidates, the method _component_match is called on each candidate to restrict further the match.

AbstractComponent are never returned by the dispatch mechanism, they can be used to share code in concrete components.

Next?

Maybe you are wondering what is "Connector" then? Maybe I should write about it one day, but in a few words: the Connector module is a set of pre-defined components specialized to implement connectors with external services.

Thank you for reading this far!

Forem: Guewen Baconnier