Fastlane-the way to automate mobile development

Fastlane-the way to automate mobile development

This article is a summary of the sharing of the "Mobile Frontline" group on August 11th. Please indicate that it is from the "Mobile Development Frontline" official account.

Guest introduction

Xing Tianyu, with 8 years of R&D and management experience, served as a front-end engineer and back-end engineer (Ruby) in the early stage, and began to get involved in mobile terminals (MeeGo, Andriod, iOS) in the early 10s. Passionate about open source, advocating efficiency. At present, he is the technical person in charge of [Gengmei], responsible for R&D team management and mobile terminal infrastructure work.

I have always believed that in the world of programs, all repetitive and process-oriented tasks can be completed by automation.

The same is true in mobile development: in fact, writing code is only part of our development process. In addition, we also need to compile, package, upload, deploy, library management, version control and other chores other than Coding, but it is These tedious and repetitive tasks take up our precious time.

Therefore, in a world of engineers with "lazy people", there will always be people trying their best to make changes, so these "lazy people" are willing to create many wonderful wheels that not only facilitate themselves, but also help others, let this The world has become a better place.

Today I will introduce you to one of the wheels: Fastlane, the star project on Github has so far obtained more than 10,000 Stars, and there are more than 1,500 Forks.

Promotion and application of Fastlane in our team

How, does it sound awesome? But don't worry, before getting into the topic, I want to briefly share with you some of our mobile team's experience in continuous testing and continuous delivery.

As everyone knows, in recent years, with the popularization of smart phones, the mobile terminal has to not only carry the realization of more business scenarios, but also respond to changing business needs. This requires our mobile team to respond quickly to changes and iterate quickly. Then the question that follows is how to ensure that the speed is increased as much as possible without sacrificing quality. I think all this needs to be built on a high-quality continuous testing and continuous delivery system.

However, the rise of the mobile terminal itself is relatively short, and various aspects of maturity are also lacking, and there are few tools that can be used. As the depth and breadth of the business increase, the iteration speed increases, such as certificate management, Packaging, uploading, and publishing such repetitive and non-technical work gradually took up everyone's time, and the team was criticized for it.

Therefore, our architecture team has been looking for such a tool and a solution since the beginning of last year to completely free the "hands" of engineers.

At the beginning, we tried to use Jenkins+Fir to build a continuous testing environment. The process is as follows:

To be honest, the effect is still possible, at least within a certain period of time to meet our requirements, but Jenkins itself is just a general CI process management system, itself does not provide such as ITC bag and Meta content management, signature, certificate management, etc. The scenario is closely integrated with the mobile terminal business, and the configuration process is quite cumbersome.

At the end of last year, by chance, we found Fastlane on Github. After reading the Readme, we felt something was interesting, so we decided to give it a try. In fact, at the beginning, we only used Fastlane to solve the problem of certificate synchronization and uploading ITC in the iOS team. However, with in-depth research, we found that Fastlane can actually do more, so we gradually applied it to the iOS side. Many scenarios, such as: private Pod release, static code inspection, UIAutomation testing, etc., and then promoted to the Andriod platform to complete a series of tasks such as the release of private AAR, Monkey testing, etc. Now it can be said that Fastlane has become an inseparable part of our work.

In addition, Fastlane itself can also be well integrated with mainstream CI systems such as Jenkins and Circle, and since the main CI processes are managed and executed by Fastlane, the complexity of these system configurations is fundamentally reduced.

Introduction to Fastlane

Having said all that, let s go back to today s protagonist. 1. let s briefly introduce: Fastlane is a set of automation tools and frameworks written in Ruby language. Each tool actually corresponds to a Ruby script to execute a specific one. The Fastlane core framework allows users to combine different tools organically and flexibly in the form of similar configuration files to form a complete automated process.

So far, Fastlane's tool set contains about 170 gadgets, which basically cover the content involved in mobile development such as packaging, signing, testing, deployment, release, library management, and so on.

The description and use of these tools can be found here:

https://docs.fastlane.tools/actions/Actions/

If these tools still don't meet your needs, it doesn't matter. Thanks to Fastlane's powerful Action and Plugin mechanisms, if you happen to know some Ruby development, you can easily write the tools you want.

In fact, about half of the official tools are produced, and the rest are contributed by members of the Github community. I am fortunate to have contributed one of them. (It is recommended that mobile development engineers still need to learn a scripting language, such as Ruby)

The installation of Fastlane is very simple. Like Cocoapods, Fastlane can also be installed via RubyGems. If you have a Ruby environment on your computer, you only need to execute the following commands to complete:

gem install fastlane 

Common scenarios and pain points of continuous testing and continuous delivery of mobile clients

Fastlane itself can do many things, but one of the most important functions is that it can be seamlessly embedded in continuous testing and continuous delivery systems.

Below, in order to make it easier for everyone to understand, I take iOS as an example and cite a few scenarios that we often encounter in the mobile development process:

scene one

When an iterative development test is over and the server is online, we will use Testflight for online follow-up testing. Generally, the following process will be performed:

  1. Execute the Git Pull command to pull the latest code to the local

  2. Pod Install installs the latest dependency libraries

  3. Increase Build Version in Xcode

  4. Click Archive in Xcode to compile and package

  5. Choose to output an ipa file in iOS AppStore mode

  6. Upload IPA to ITC (TestFlight) via Application Loader

  7. Then wait for the completion of the ITC Process, log in and select the Build just now for TestFlight testing

  8. Since the version number has been modified, the code Commit and Push are required

If there is a problem in the online follow-up test, you need to repeat the above 8 steps after the repair is completed. In fact, the students who have done this should have experienced it. If it goes smoothly, it will take about 30 minutes at a time. If you forget to increase the Build Version at a certain time, then the previous work will be done in vain. In the early days of our team, because the automation system had not yet been established, we had a colleague who was specifically responsible for this. During the two days of online testing, he had half a day to do nothing else. Basically, he was uploading the package and saying it. All tears.

Scene two

With the development of business and the increase of product lines, we need to split the APP into several basic components and business components for cross-APP use and convenient management and maintenance (this is another big issue, so I won t repeat it here. ). Each component is managed by a private Pod. Pod release and update have also become part of our daily work. For these Pods, the general internal principles of our team are: who makes, who manages, who publishes, and the person in charge of the pod We call it the library management internally, and this matter is also shared by each library management. The process for the library management to release a Pod is approximately as follows:

  1. Increase the version number in Podspec

  2. Execute pod lib lint command for library verification

  3. Git Commit code

  4. Git Push code to the remote

  5. Make a Git Tag

  6. Push Tag to the remote

  7. Execute the pod repo push command to publish the library to the private warehouse

If there are only two or three libraries, and the update frequency of the library is low, it is okay to handle it manually each time. But when the number of libraries gradually increases, this matter becomes quite troublesome, especially when the top-level library depends on the bottom-level library, then upgrading a library will have far more impact than itself. If it is handled manually, the entire The process can become quite painful.

Having said that, I believe that all students who have done this should feel deeply about it.

So let's take a look at how to use Fastlane to solve the above two scenarios.

In fact, it is not difficult to say, first execute under the project:

fastlane init 

Then follow the configuration guide, fill in the App and ITC related information, and then Fastlane will create a fastlane directory in the project directory, which contains all the configuration related to this project, and the rest is to configure the above process in the fastlane directory In Fastfile,

Fastfile of scenario one (the HipChat part can be ignored): 

The Fastfile of the second scenario (the HipChat part can be ignored): 

For the configuration of Fastlane, there will be a more detailed description in the official documentation:

https://docs.fastlane.tools/getting-started/ios/setup/

After configuring this script, the rest is very easy. For scenario one, we can execute the following commands with the terminal in the project directory:

fastlane do_deliver_app 
project:Gengmei 
scheme:Gengmei-AppStore 
version:6.3.0 
build:201609011530 
... 

In the same way, for scenario 2, we can execute the following commands in the project directory with the terminal:

fastlane do_release_lib project:GMUtil version:0.1.4 

For any complicated process, basically one command is completed, is it very convenient?

In addition, in order to make Fastlane's various commands more foolish and visual, we have developed a system suitable for continuous testing and continuous release of mobile terminal Jaguar based on the Fastlane kernel and using the Ruby on Rails framework. Jaguar itself connects with Gitlab, Sentry, Jira, HipChat, Maven and other internal systems to form a complete automation system.

In this way, for most of the automated processes, Jaguar will be automatically triggered through various timed tasks or WebHook; a small number of manual operations are required, and engineers only need to click a button to complete.

Here is a screenshot of Jaguar: 

The feeling of using Fastlane, a simple review of stepping on the pit

After using Fastlane for such a long time, my deepest feeling is: Fastlane truly liberates engineers from all kinds of boring but necessary repetitive labor and process work, focusing on the business or architecture itself, making the entire development Efficiency, test efficiency, and operation and maintenance efficiency are greatly improved.

In the process of using Fastlane, there are some small precautions, also share with you:

Since Fastlane itself has a relatively high update frequency, about once every 1-2 weeks, if the latest version has not been upgraded, it is recommended to read the Release Notes of these updates carefully, otherwise there may be some strange bugs. For example: When upgrading from 1.87 version to 1.88, if you do not add the following two lines in the Fastfile:

ENV['FASTLANE_EXPERIMENTAL_TRANSPORTER_AVOID_SHELL_SCRIPT'] = '1'
ENV['SPACESHIP_LOGIN_ENCODING_IDENTITY'] = '1' 

Then, when uploading ITC, an error will be reported, and the specific reason cannot be seen from the error. Finally, after various tossings, I finally found the reason in the issue on Github: the spaceship tool added a small feature. Bug.

In addition, if you encounter a problem while using Fastlane, it is recommended to directly submit an issue on Github. The authors and community engineers of Fastlane will respond very quickly and will be very enthusiastic to help you solve the problem. Of course, everyone When asking questions, please describe as clearly as possible, the code should be posted, the screenshot of the screenshot.

Divergent thinking, Fastlane is not only exclusive to mobile

Although Fastlane itself was born for mobile, if we use our imagination, we will find that it is also possible to sort out the continuous integration and continuous delivery processes of the back-end and front-end, and then unify them to Fastlane to manage ( Of course, some Plugin or Action needs to be customized in this process)

For example: Our team is currently planning to integrate the UI automation test process of the PC station first:

  1. Execute the Git Pull command to pull the latest code

  2. Install Requirements using pip

  3. Confused and compressed front-end Javascript and CSS

  4. Restart Django service

  5. Use Selenium to perform UI automation testing

  6. Collect test results and send email to QA team

The corresponding Fastfile is as follows (abbreviated, not really used yet): 

In this way, the continuous integration and continuous delivery of the client, front-end, and back-end within a team can be deployed, managed, and maintained uniformly, so that the infrastructure can meet the team's demand for automation as much as possible. Why not do it? It.

Conclusion

This sharing is just to give you a taste of Fastlane's style, for details such as: how to customize Action, Plugin, how to use details on the Andriod platform, how to apply it in automated test scenarios, and some advanced usage of Fastlane, etc., I will In the next period of time, make the corresponding sorting out and form an article for your reference.

Due to my limited level, mistakes and omissions are unavoidable. I welcome all students to correct me. If you have a better case in the use of Fastlane, you are also welcome to exchange and share.

Finally, attach a Fastfile script and some custom Actions that our team is using: https://github.com/GengmeiRD/Fastfiles

In addition, Fastlane also provides some examples of foreign teams: https://github.com/fastlane/examples

QA excerpt

Q: Testflight is very unstable, why switch from FIR to Testflight?

A: I have used it for so long, and I feel that Testflight is still relatively stable. In addition, we still use Fir in the test environment. Testflight will only be used during online follow-up testing in the production environment (it can be understood as pre-release).

Q: Fastlane executes multiple commands at once. If the command fails to execute, how to quickly locate the problem or determine which command is wrong? There is a log parsing service in your architecture diagram. Is it related to this?

A: Under normal circumstances, you can see the corresponding log, which will have a detailed description of the error.

Q: In large-scale mobile projects, plug-in development is generally involved. Of course, many companies use modules to upload maven and reference the main project. Either way, it will involve the collaboration of multiple projects. For example, if we want to issue a package, we need to update the code of each project, then the plug-in project is compiled and issued, and finally the main project is compiled and issued. Can Fastlane handle this situation?

A: We are also considering this. What we can do at present is the release of aar files.

Q: I encountered some problems when writing some actions when doing Fastfile. For example, writing latest_testflight_build_number always reported an error. Is there a way to catch the error? Do you want to learn Ruby language?

A: Some Ruby basics are needed. I suggest you learn it. Ruby itself is not difficult.

Q: Does the project have to use pods? Will there be problems if pods are not used?

A: No, if it is managed by carthage, it can be the same.

Q: Have you tried to use React Native? Will there be a problem?

A: Although I haven't used it before, in principle, there will be no problem

Q: Is Jaguar open source and what is the github address?

A: It is planned, but the current maturity has not yet reached my expectations. In addition, some actual businesses need to be divested and open sourced before the end of the year.

Q: Do you have any suggestions or ideas if you want to build a publishing platform by yourself? How much does it cost to build such a platform?

A: It is recommended to write directly in Ruby, which can be seamlessly integrated with Fastlane. We invested about one and a half engineers in establishing Jaguar, which took less than a month.

Q: The number of servers and configuration requirements?

A: We used a Mac Mini to get it done.

Q: Did you use a mac mini exclusively as a server in your actual application? What is the amount of resources occupied in fastlane work? How many times do you run fastlane scripts a day? A small company can use a mac for programmer s development and use part of its resources as a server, right?

A: Yes, a Mac Mini, Fastlane itself has very little overhead, mainly because commands such as xcbuild consume more resources, we run about 20-25 times a day. There is no problem with a personal Mac.

Q: How long does it take for your iOS project to be compiled from the beginning to when the version is received by QA?

A: It depends on the size of the project. It usually takes 5 minutes to 10 minutes. Android will be faster.

Scan the QR code to follow the front lines of mobile development and find more good articles on mobile development!