diff options
Diffstat (limited to 'app/serializers/README.md')
| -rw-r--r-- | app/serializers/README.md | 325 |
1 files changed, 325 insertions, 0 deletions
diff --git a/app/serializers/README.md b/app/serializers/README.md new file mode 100644 index 00000000000..0337f88db5f --- /dev/null +++ b/app/serializers/README.md @@ -0,0 +1,325 @@ +# Serializers + +This is a documentation for classes located in `app/serializers` directory. + +In GitLab, we use [grape-entities][grape-entity-project], accompanied by a +serializer, to convert a Ruby object to its JSON representation. + +Serializers are typically used in controllers to build a JSON response +that is usually consumed by a frontend code. + +## Why using a serializer is important? + +Using serializers, instead of `to_json` method, has several benefits: + +* it helps to prevent exposure of a sensitive data stored in the database +* it makes it easier to test what should and should not be exposed +* it makes it easier to reuse serialization entities that are building blocks +* it makes it easier to move complexity from controllers to easily testable + classes +* it encourages hiding complexity behind intentions-revealing interfaces +* it makes it easier to take care about serialization performance concerns +* it makes it easier to reduce merge conflicts between CE -> EE +* it makes it easier to benefit from domain driven development techniques + +## What is a serializer? + +A serializer is a class that encapsulates all business rules for building a +JSON response using serialization entities. + +It is designed to be testable and to support passing additional context from +the controller. + +## What is a serialization entity? + +Entities are lightweight structures that allow to represent domain models +in a consistent and abstracted way, and reuse them as building blocks to +create a payload. + +Entities located in `app/serializers` are usually derived from a +[`Grape::Entity`][grape-entity-class] class. + +Serialization entities that do require to have a knowledge about specific +elements of the request, need to mix `RequestAwareEntity` in. + +A serialization entity usually maps a domain model class into its JSON +representation. It rarely happens that a serialization entity exists without +a corresponding domain model class. As an example, we have an `Issue` class and +a corresponding `IssueSerializer`. + +Serialization entites are designed to reuse other serialization entities, which +is a convenient way to create a multi-level JSON representation of a piece of +a domain model you want to serialize. + +See [documentation for Grape Entites][grape-entity-readme] for more details. + +## How to implement a serializer? + +### Base implementation + +In order to effectively implement a serializer it is necessary to create a new +class in `app/serializers`. See existing serializers as an example. + +A new serializer should inherit from a `BaseSerializer` class. It is necessary +to specify which serialization entity will be used to serialize a resource. + +```ruby +class MyResourceSerializer < BaseSerialize + entity MyResourceEntity +end +``` + +The example above shows how a most simple serializer can look like. + +Given that the entity `MyResourceEntity` exists, you can now use +`MyResourceSerializer` in the controller by creating an instance of it, and +calling `MyResourceSerializer#represent(resource)` method. + +Note that a `resource` can be either a single object, an array of objects or an +`ActiveRecord::Relation` object. A serialization entity should be smart enough +to accurately represent each of these. + +It should not be necessary to use `Enumerable#map`, and it should be avoided +from the performance reasons. + +### Choosing what gets serialized + +It often happens that you might want to use the same serializer in many places, +but sometimes the intention is to only expose a small subset of object's +attributes in one place, and a different subset in another. + +`BaseSerializer#represent(resource, opts = {})` method can take an additional +hash argument, `opts`, that defines what is going to be serialized. + +`BaseSerializer` will pass these options to a serialization entity. See +how it is [documented in the upstream project][grape-entity-only]. + +With this approach you can extend the serializer to respond to methods that will +create a JSON response according to your needs. + +```ruby +class PipelineSerializer < BaseSerializer + entity PipelineEntity + + def represent_details(resource) + represent(resource, only: [:details]) + end + + def represent_status(resource) + represent(resource, only: [:status]) + end +end +``` + +It is possible to use `only` and `except` keywords. Both keywords do support +nested attributes, like `except: [:id, { user: [:id] }]`. + +Passing `only` and `except` to the `represent` method from a controller is +possible, but it defies principles of encapsulation and testability, and it is +better to avoid it, and to add a specific method to the serializer instead. + +### Reusing serialization entities from the API + +Public API in GitLab is implemented using [Grape][grape-project]. + +Under the hood it also uses [`Grape::Entity`][grape-entity-class] classes. +This means that it is possible to reuse these classes to implement internal +serializers. + +You can either use such entity directly: + +```ruby +class MyResourceSerializer < BaseSerializer + entity API::Entities::SomeEntity +end +``` + +Or derive a new serialization entity class from it: + +```ruby +class MyEntity < API::Entities::SomeEntity + include RequestAwareEntity + + unexpose :something +end +``` + +It might be a good idea to write specs for entities that do inherit from +the API, because when API payloads are changed / extended, it is easy to forget +about the impact on the internal API through a serializer that reuses API +entities. + +It is usually safe to do that, because API entities rarely break backward +compatibility, but additional exposure may have a performance impact when API +gets extended significantly. Write tests that check if only necessary data is +exposed. + +## How to write tests for a serializer? + +Like every other class in the project, creating a serializer warrants writing +tests for it. + +It is usually a good idea to test each public method in the serializer against +a valid payload. `BaseSerializer#represent` returns a hash, so it is possible +to use usual RSpec matchers like `include`. + +Sometimes, when the payload is large, it makes sense to validate it entirely +using `match_response_schema` matcher along with a new fixture that can be +stored in `spec/fixtures/api/schemas/`. This matcher is using a `json-schema` +gem, which is quite flexible, see a [documentation][json-schema-gem] for it. + +## How to use a serializer in a controller? + +Once a new serializer is implemented, it is possible to use it in a controller. + +Create an instance of the serializer and render the response. + +```ruby +def index + format.json do + render json: MyResourceSerializer + .new(current_user: @current_user) + .represent_details(@project.resources) + nd +end +``` + +If it is necessary to include additional information in the payload, it is +possible to extend what is going to be rendered, the usual way: + +```ruby +def index + format.json do + render json: { + resources: MyResourceSerializer + .new(current_user: @current_user) + .represent_details(@project.resources), + count: @project.resources.count + } + nd +end +``` + +Note that in these examples an additional context is being passed to the +serializer (`current_user: @current_user`). + +## How to pass an additional context from the controller? + +It is possible to pass an additional context from a controller to a +serializer and each serialization entity that is used in the process. + +Serialization entities that do require an additional context have +`RequestAwareEntity` concern mixed in. This piece of the code exposes a method +called `request` in every serialization entity that is instantiated during +serialization. + +An object returned by this method is an instance of `EntityRequest`, which +behaves like an `OpenStruct` object, with the difference that it will raise +an error if an unknown method is called. + +In other words, in the previous example, `request` method will return an +instance of `EntityRequest` that responds to `current_user` method. It will be +available in every serialization entity instantiated by `MyResourceSerializer`. + +`EntityRequest` is a workaround for [#20045][issue-20045] and is meant to be +refactored soon. Please avoid passing an additional context that is not +required by a serialization entity. + +At the moment, the context that is passed to entities most often is +`current_user` and `project`. + +## How is this related to using presenters? + +Payload created by a serializer is usually a representation of the backed code, +combined with the current request data. Therefore, technically, serializers +are presenters that create payload consumed by a frontend code, usually Vue +components. + +In GitLab, it is possible to use [presenters][presenters-readme], but +`BaseSerializer` still needs to learn how to use it, see [#30898][issue-30898]. + +It is possible to use presenters when serializer is used to represent only +a single object. It is not supported when `ActiveRecord::Relation` is being +serialized. + +```ruby +MyObjectSerializer.new.represent(object.present) +``` + +## Best practices + +1. Do not invoke a serializer from within a serialization entity. + + If you need to use a serializer from within a serialization entity, it is + possible that you are missing a class for an important domain concept. + + Consider creating a new domain class and a corresponding serialization + entity for it. + +1. Use only one approach to switch behavior of the serializer. + + It is possible to use a few approaches to switch a behavior of the + serializer. Most common are using a [Fluent Interface][fluent-interface] + and creating a separate `represent_something` methods. + + Whatever you choose, it might be better to use only one approach at a time. + +1. Do not forget about creating specs for serialization entities. + + Writing tests for the serializer indeed does cover testing a behavior of + serialization entities that the serializer instantiates. However it might + be a good idea to write separate tests for entities as well, because these + are meant to be reused in different serializers, and a serializer can + change a behavior of a serialization entity. + +1. Use `ActiveRecord::Relation` where possible + + Using an `ActiveRecord::Relation` might help from the performance perspective. + +1. Be diligent about passing an additional context from the controller. + + Using `EntityRequest` and `RequestAwareEntity` is a workaround for the lack + of high-level mechanism. It is meant to be refactored, and current + implementation is error prone. Imagine the situation that one serialization + entity requires `request.user` attribute, but the second one wants + `request.current_user`. When it happens that these two entities are used in + the same serialization request, you might need to pass both parameters to + the serializer, which is obviously not a perfect situation. + + When in doubt, pass only `current_user` and `project` if these are required. + +1. Keep performance concerns in mind + + Using a serializer incorrectly can have significant impact on the + performance. + + Because serializers are technically presenters, it is often necessary + to calculate, for example, paths to various controller-actions. + Since using URL helpers usually involve passing `project` and `namespace` + adding `includes(project: :namespace)` in the serializer, can help to avoid + N+1 queries. + + Also, try to avoid using `Enumerable#map` or other methods that will + execute a database query eagerly. + +1. Avoid passing `only` and `except` from the controller. +1. Write tests checking for N+1 queries. +1. Write controller tests for actions / formats using serializers. +1. Write tests that check if only necessary data is exposed. +1. Write tests that check if no sensitive data is exposed. + +## Future + +* [Next iteration of serializers][issue-27569] + +[grape-project]: http://www.ruby-grape.org +[grape-entity-project]: https://github.com/ruby-grape/grape-entity +[grape-entity-readme]: https://github.com/ruby-grape/grape-entity/blob/master/README.md +[grape-entity-class]: https://github.com/ruby-grape/grape-entity/blob/master/lib/grape_entity/entity.rb +[grape-entity-only]: https://github.com/ruby-grape/grape-entity/blob/master/README.md#returning-only-the-fields-you-want +[presenters-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/presenters/README.md +[fluent-interface]: https://en.wikipedia.org/wiki/Fluent_interface +[json-schema-gem]: https://github.com/ruby-json-schema/json-schema +[issue-20045]: https://gitlab.com/gitlab-org/gitlab-ce/issues/20045 +[issue-30898]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30898 +[issue-27569]: https://gitlab.com/gitlab-org/gitlab-ce/issues/27569 |