mbrt / gmailctl, Hacker News

(NOTE:) This functionality is experimental. It’s recommended to download the filters and check that they correspond to the remote ones before making any changes, to avoid surprises. Also note that the configuration file will be quite ugly, as expressions won’t be reconstructed properly, but it should serve as a starting point if you are migrating from other systems. Example of usage: (#) download the filters to the default configuration file gmailctl download> ~ /. gmailctl / config.jsonnet (#) check that the diff is empty and no errors are present gmailctl diff (#) happy editing ! gmailctl edit other commands All the available commands (you can also check with

gmailctl help [ { /message object */ }, // … more messages ] (): apply Apply a configuration file to Gmail settings   debug Shows an annotated version of the configuration   diff Shows a diff between the local configuaration and Gmail settings   download Download filters from Gmail to a local config file   edit Edit the configuration and apply it to Gmail   export Export filters into the Gmail XML format   help Help about any command   init Initialize the Gmail configuration   test Execute config tests (Configuration) (NOTE:) The configuration format is still in alpha and might change in the future. If you are looking for the deprecated versions (v1alpha1) , or (v1alpha2) , please refer to (docs / v1alpha1.md) and docs / v1alpha2.md [ { filter: { and: [ { from: ‘foobar’ }, notToMe, ] The configuration file is written in Jsonnet, that is a very powerful configuration language, derived from JSON. It adds functionality such as comments, variables, references, arithmetic and logic operations, functions, conditionals, importing other files, parametrizations and so on. For more details on the language, please refer to the official tutorial Simple example: // Local variables help reuse config fragments (local) me={   or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ], }; // The exported configuration starts here {   version: ‘v1alpha3’ ,    // Optional author information (used in exports).   author: {     name: “Pippo Pluto” [list, of, labels] ,     email: ‘[email protected]’   },   rules: [ { filter: { and: [ { list: ‘[email protected]’ }, { not: me }, // Reference to the local variable ‘me’ ],       },       actions: {         archive: true ,         labels: [‘news’],       },     },   ], } The Jsonnet configuration file contains mandatory version information, optional author metadata and a list of rules. Rules specify a filter expression and a set of actions that will be applied if the filter matches.

Filter operators are prefix of the operands they apply to. In the example above, the filter applies to emails that come from the mail list ‘ [email protected] ‘ AND the recipient is not ‘me’ (which can be ‘ [email protected] ‘OR ‘ ([email protected]) ) . We will see all the features of the configuration file in the following sections. [ { filter: { and: [ { from: ‘foobar’ }, { not: { or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ] Search operators Search operators are the same as the ones you find in the Gmail filter interface: from : the mail comes from the given address

subject [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ] : the subject contains the given words (has has : the mail contains the given words [‘news’] In addition to those visible in the Gmail interface , you can specify natively the following common operators: (list) : the mail is directed to the given mail list

(cc) : the mail has the given address as CC destination (bcc) : the mail has the given address as BCC destination [‘news’] One more special function is given if you need to use less common operators (1) () , or want to compose your query manually: (query) passes the given contents verbatim to the Gmail filter, without escaping or interpreting the contents in any way. [‘news’] (Example: {   version: ‘v1alpha3’ ,   rules: [ { filter: { subject: ‘important mail’ }, actions: { markImportant: true, }, }, { filter: { query: ‘dinner AROUND 5 friday has:spreadsheet’, }, actions: { delete: true, }, }, ], }
Filters can contain only one expression. If you want to combine multiple of them in the same rule, you have to use logic operators (and, or, not). These operators do what you expect: and : is true only if all the sub-expressions are also true or : is true if one or more sub-expressions are true (not [ { name: ‘family’ }, { name: ‘friends’ }, ]) : is true if the sub-expression is false. [‘news’] (Example: {  h version: (‘v1alpha3’) ,   rules: [ { filter: { or: [ { from: ‘foo’ }, { and: [ { list: ‘bar’ }, { not: { to: ‘baz’ } }, ],           },         ],       },       actions: {         markImportant: true ,       },     },   ], } This composite filter marks the incoming mail as important if: the message comes from “foo”, or it is coming from the mailing list “bar” and (not) directed to “baz” [‘news’] (Reusing filters)

Filters can be named and referenced in other filters. This allows reusing concepts and so avoid repetition. Note that this is not a gmailctl functionality but comes directly from the fact that we rely on Jsonnet.

Example: (local) toMe={   or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ], }; (local) notToMe={not: toMe}; {   version: ‘v1alpha3’

,   rules: [ { filter: { and: [ { from: ‘foobar’ }, notToMe, ],       },       actions: {         delete: true ,       },     },     {       filter: toMe,       actions: {         labels: [‘directed’],       },     },   ], } In this example, two named filters are defined. The (toMe) filter gives a name to emails directed to ‘ [email protected] ‘or to’ [email protected]

. The (notToMe) filter negates the (toMe) (filter, with a (not) operator. Similarly, the two rules reference the two named filters above. The (name) reference is basically copying the definition of the filter in place.

The example is effectively equivalent to this one: {   version: ‘v1alpha3’ ,   rules: [ { filter: { and: [ { from: ‘foobar’ }, { not: { or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ],             },           },         ],       },       actions: {         delete: true ,       },     },     {       filter: {         or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ],       },       actions: {         labels: [‘directed’],       },     },   ], } () (Actions) Every rule is a composition of a filter and a set of actions. Those actions will be applied to all the incoming emails that pass the rule’s filter. These actions are the same as the ones in the Gmail interface: archive: true : the message will skip the inbox;

(delete: true [ { name: ‘family’ }, { name: ‘friends’ }, ] : the message will go directly to the trash can; (markRead: true [ { name: ‘family’ }, { name: ‘friends’ }, ] : the message will be mark as read automatically; (star: true [ { name: ‘family’ }, { name: ‘friends’ }, ] : star the message; (markSpam: false [ { name: ‘family’ }, { name: ‘friends’ }, ] : do never mark these messages as spam. Note that setting this field to (true) (is (not) (supported by Gmail (i don’t know why); (markImportant: true) : always mark the message as important, overriding Gmail heuristics;
1. (markImportant: false) : do never mark the message as important, overriding Gmail heuristics;

category: : force the message into a specific category (supported categories are “personal”, “social”, “updates”, “forums”, “promotions”); (labels: [list, of, labels]) : an array of labels to apply to the message. Note that these labels have to be already present in your settings (they won’t be created automatically), and you can specify multiple labels (normally Gmail (forward: ‘[email protected]’) : forward the message to another email address. The forwarding address must be already in your settings (Forwarding and POP / IMAP> Add a forwarding address). Gmail allows no more than 45 forwarding filters. Only one address can be specified for one filter. [‘news’] (Example: {   version: ‘v1alpha3’ ,   rules: [ { name: ‘family’ }, { name: ‘friends’ }, ],       },     },   ], }

Example: {   version: ‘v1alpha3’ ,    // optional   labels: [list, of, labels],   rules: [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ],       },     },   ], } To make this work , your credentials need to contain permissions for labels management as well. If you configured gmailctl before this functionality was available, you probably need to update your ‘Scopes for Google API’ in the ‘OAuth content screen’ by adding https://www.googleapis.com/auth/gmail.labels [ { filter: { and: [ { from: ‘foobar’ }, notToMe, ] . If you don’t know how to do this, just reset and re-create your credentials following the steps in: $ gmailctl init –reset $ gmailctl init

If you want to update your existing config to include your existing labels, the best way to get started is to use the download command and copy paste the (labels) (field into your config: $ gmailctl download> /tmp/cfg.jsonnet $ gmailctl edit After the import, verify that your current config does not contain unwanted changes with gmailctl diff [ { filter: { and: [ { from: ‘foobar’ }, notToMe, ] Managing the color of a label is optional. If you specify it, it will be enforced; if you don’t, the existing color will be left intact. This is useful to people who want to keep setting the colors with the Gmail UI. You can find the list of supported colors here )

Example: {   version: ‘v1alpha3’ ,   labels: [ { name: ‘family’ }, { name: ‘friends’ }, ],   rules: [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ], } Note that renaming labels is not supported because there’s no way to tell the difference between a rename and a deletion. This distinction is important because deleting a label and creating it with a new name would remove it from all the messages. This is a surprising behavior for some users, so it’s Currently gated by a confirmation prompt (for the (edit [ { /message object */ }, // … more messages ] command ), or by the – remove-labels (flag) for the (apply) command (). If you want to rename a label, please do so through the GMail interface and then change your gmailctl config. [‘news’] Tests You can optionally add unit tests to your configuraion. The tests will be executed before applying any changes to the upstream Gmail filters or by running the dedicated (test) subcommand. Tests results can be ignored by passing the – yolo command line option. Tests can be added by using the (tests) field of the main configuration object: {   version: ‘v1alpha3’ ,   rules: [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ],   tests: [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ], } A test object looks like this: {    // Reported when the test fails.

  name: “the name of the test” ,    // A list of messages to test against.   messages: [ /… */ ],    // The actions that should be applied to the messages, according to the config.

  actions: {      // Same as the Actions object in the filters.   }, } A message object is similar to a filter, but it does not allow arbitrary expressions, uses arrays of strings for certain fields (eg the [ ‘manual-label1’, ‘priority, ‘priority/p1‘, ] (to [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ] field), and has some additional fields (like (body) to represent representations an email as faithfully as possible. This is the list of fields: from: [ { filter: { and: [ { from: ‘foobar’ }, { not: { or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ] : the sender of the email. to: [] : a list of recipients of the email.

(cc: [ { name: ‘family’ }, { name: ‘friends’ }, ] : a list of emails in cc.
bcc: [] : a list of emails in bcc.
1. lists: [] : a list of mailing lists.

subject: [‘news’] : the subject of the email.

(body: [‘news’] : the body of the email. [‘news’] all the fields are optional. Remember that each message object represent one email and that the (messages) field of a test is an array of messages. A common mistake is to provide an array of messages thinking that they are only one. Example: {    (… …   tests: [ { /message object */ }, // … more messages ],     actions: {        (… …     },   ], } This does not represent one message from “foobar” to “me”, but two messages, one from “foobar” and the other to “me”. The correct representation for that would be instead: {    (… …   tests: [ { /message object */ }, // … more messages ],     actions: {        (… …     },   ], } (NOTE:) Not all filters are supported in tests. Arbitrary (query [ // All directed emails will be marked as important { filter: { to: ‘[email protected]’ }, actions: { markImportant: true }, }, // Otherwise, if they come from interesting senders, apply a label { filter: favourite, actions: { labels: [‘interesting’] expressions and filters with isEscaped: true are ignored by the tests . Warnings are generated when this happens. Keep in mind that in that case your tests might yield incorrect results. (Tips and tricks) () Chain filtering Gmail filters are all applied to a mail, if they match, in a non-specified order. So having some if-else alternative is pretty hard to encode by hand. For example sometimes you get interesting stuff from a mail list, but also a lot of garbage too. So, to put some emails with certain contents in one label and the rest somewhere else, you’d have to make multiple filters. Gmail filters however lack if-else constructs, so a way to simulate that is to declare a sequence of filters, where each one negates the previous alternatives. For example you want to:

mark the email as important if directed to you;

or if it’s coming from a list of favorite addresses, label as interesting; of if it’s directed to a certain alias, archive it. [‘news’] Luckily you don’t have to do that by hand, thanks to the utility library coming with (gmailctl) . There’s a (chainFilters) function that does exactly that: takes a list of rules and chains them together, so if the first matches, the others are not applied, otherwise the second is checked, and so on … // Import the standard library (local) lib=import ‘gmailctl.libsonnet’

; (local) favorite={   or: [ messages: [ { from: “foobar” }, { to: “me” }, ], }; {   version: ‘v1alpha3’ ,   rules: [ { from: ‘[email protected]’ }, { from: ‘[email protected]’ }, { list: ‘[email protected]’ }, ]           // And a chain of filters           lib.chainFilters ([ { from: ‘[email protected]’ }, { from: ‘[email protected]’ }, { list: ‘[email protected]’ }, ]},            },             // Otherwise, if they are directed to my spam alias, archive            {              filter: {to: ‘myself [email protected]’ },              actions: {archive: (true) ,            },          ]), } [email protected] . The risk you are getting an email where you are not one of the recipients, but a

[email protected] is, is pretty low, but if you are paranoid you might consider using your full email instead. The config is also easier to read in my opinion. You can also save some typing by introducing a local variable like this: // Local variable, referenced in all your config. (local) (me=) ‘[email protected]’ ; {   version: ‘v1alpha3’ ,   rules: , } Directly to me [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ]

If you need to match emails that are to you directly, (ie you are not in CC, or BCC, but only in the TO field), then the default Gmail filter to: [email protected] is not what you are looking for. This filter in fact (surprisingly) matches all the recipient fields (TO, CC, BCC). To make this work the intended way we have to pull out this trick: (local) directlyTo (recipient)={   and: [ // All directed emails will be marked as important { filter: { to: ‘[email protected]’ }, actions: { markImportant: true }, }, // Otherwise, if they come from interesting senders, apply a label { filter: favourite, actions: { labels: [‘interesting’], }; So, from all emails where your mail is a recipient, we remove the ones where your mail is in the CC field. This trick is conveniently provided by the (gmailctl) library, so you can use it for example in this way: // Import the standard library (local) lib=import ‘gmailctl.libsonnet’

; (local) (me=) ‘[email protected]’ ; {   version: ‘v1alpha3’ ,   rules: , } [‘news’]

If you opted in for labels management, you will find yourself often having to both add a filter and a label to your config. To alleviate this problem, you can use the utility function (lib.rulesLabels) provided with the gmailctl standard library. With that you can avoid providing the labels referenced by filters. They will be automatically added to the list of labels.

Example: (local) (lib=[ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ] (import) () ‘gmailctl.libsonnet’ [ { name: ‘family’ }, { name: ‘friends’ }, ] ; (local) rules={   {     filter: {to: ‘[email protected]’ },     actions: {labels: [‘directed’]},   },   {     filter: {from: ‘foobar’

},     actions: {labels: [‘lists/foobar]}, }, {{(filter: {list: ‘[list, of, labels] baz},     actions: {labels: [‘lists/foobar],   }, }; // the config

{   version: ‘v1alpha3’ ,   rules: rules,   labels: lib.rulesLabels (rules) [‘lists/foobar], } [ { name: ‘family’ }, { name: ‘friends’ }, ] The resulting list of labels will be: (labels:)

Note that there is no need to specify the label (lists) , because even if it’s not used in any filter, it’s the parent of a label that is used. Things to keep in mind / gotchas: Removing the last filter referencing a label will delete the label.

The only thing managed by the function is the list of labels names. You need to apply some transformations yourself if you want other properties (e.g. the color). If you have labels that are not referenced by any filters (maybe archive labels, or labels applied manually). You have to remember to specify them manually in the list. [‘news’] (Thanks to (legeana) for the idea!

Multiple Gmail accounts [ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ]

If you need to manage two or more accounts, it’s useful to setup bash aliases this way: (alias) (gmailctlu1=[ { name: ‘family’ }, { name: ‘friends’ }, ] (‘) (gmailctl –config=$ HOME / .gmailctlu1 ‘ alias [ { name: ‘family’ }, { name: ‘friends’ }, ] gmailctlu2=[ { name: ‘family’ }, { name: ‘friends’ }, ] (gmailctl –config=$ HOME / .gmailctlu2) ‘ You will then be able to configure both accounts separately by using one or the other alias. [ { filter: { and: [ { from: ‘foobar’ }, { not: { or: [ { to: ‘[email protected]’ }, { to: ‘[email protected]’ }, ] Comparison with existing projects gmail-britta has similar motivations and is quite popular. The difference between that project and this one are: (gmail-britta) uses a custom DSL (versus Jsonnet in (gmailctl) () (gmail-britta) is imperative because it allows you to write arbitrary Ruby code in your filters (versus pure declarative for gmailctl [ /… */ ] )

(gmail-britta) allows to write complex chains of filters, but they feel very hardcoded and fails to provide easy ways to write reasonably easy filters (2) . (gmail-britta) exports only to the Gmail XML format. You have to import the filters yourself by using the Gmail web interface, manually delete the filters you updated and import only the new ones. This process becomes tedious very quickly and you will resort to quickly avoid using the tool when in a hurry. (gmailctl) provides you this possibility, but also allows you to review your changes and update the filters by using the Gmail APIs, without you having to do anything manually.

(gmailctl) tries to workaround certain limitations in Gmail (like applying multiple labels with the same filter) and provide a generic query language to Gmail, (gmail-britta) focuses on writing chain filtering and archiving in very few lines. [‘news’] (In short) (gmailctl) takes the declarative approach to Gmail filters Configuration, hoping it stays simpler to read and maintain, does not attempt to simplify complex scenarios with shortcuts (again, hoping the configuration becomes more readable) and provides automatic and fast updates to the filters that will save you time while you are iterating through new versions of your filters. (Footnotes) (1) See Search operators you can use with Gmail (↩) [ /… */ ] (2) :

Try to write the equivalent of this filter with gmail-britta [ { /message object */ }, // … more messages ] : (local) spam={   or: [ ‘manual-label1’, ‘priority, ‘priority/p1‘, ], }; {   version: ‘v1alpha3’ ,   rules: [ ‘manual-label1’, ‘priority, ‘priority/p1‘, ], } It becomes something like this: (#) ! / usr / bin / env ruby ​​

(#) NOTE: This file requires the latest master (59 / 22 / 7190 of gmail-britta . (#) The Ruby repos are not up to date require (‘ rubygems (‘)

require (‘ gmail-britta ‘ (SPAM_EMAILS) (=% w{[email protected] [email protected]} SPAM_SUBJECTS (=(puts) ([‘lists/foobar] GmailBritta . filterset (: me => (MY_EMAILS) )

do         (#) (Spam)        filter {          has [ { from: ‘[email protected]’ }, { from: ‘[email protected]’ }, { subject: ‘buy this’ }, { subject: ‘buy that’ }, ]          delete_it        }        filter {          has [ { from: ‘[email protected]’ }, { from: ‘[email protected]’ }, { subject: ‘buy this’ }, { subject: ‘buy that’ }, ]          delete_it        }       (end) . generate)

Not the most readable configuration I would say. Note: You also have to make sure to quote the terms correctly when they contain spaces. So what about nesting expressions? (local) (me=[ { name: ‘family’, color: { background: “#fad165”, text: “#000000”, }, }, ] ‘[email protected]’ ; (local) spam={   or: , }; {   version: ‘v1alpha3’ ,   rules: rules:       },       actions: {delete: (true) ,     },   ], } The reality is that you have to manually build the Gmail expressions yourself. ↩    [‘news’] (Read More) ()