合并报表excel自动模板_自动合并

news/2024/7/21 4:15:06 标签: python, excel, git

合并报表excel自动模板

介绍 (Intro)

One day we decided that we don’t want to waste any more time on tasks that can be automated and we decided to start with automation of our merging process.

有一天,我们决定不想再将时间浪费在可以自动化的任务上,因此我们决定从合并过程的自动化开始。

Today’s development teams use various branching models. Ranging from the straightforward ones, such as the trunk-based model, up to a completely autonomous continuous deployment process. Many consider continuous deployment to be a utopia for large projects and only achievable for small, startup projects.

当今的开发团队使用各种分支模型。 从简单的过程(例如基于主干的模型)到完全自主的连续部署过程,一应俱全。 许多人认为,对于大型项目而言,连续部署是乌托邦,而对于小型启动项目而言,则只能实现。

We realized that with a sophisticated — multiple development loops — model, such as the one we use, automation is a necessity, not an option. As an example, at any particular moment, we might have a standard feature development loop living next to an emergency bug fixing loop (that needs to go to production asap) and a standard bug fixing loop (that can wait for two or more weeks) in addition. For various reasons, each loop is going to be deployed at some unknown time in the future. Every single loop usually comes with a dedicated branch and then developers create short-living branches that are eventually merged into this dedicated branch.

我们意识到,对于一个复杂的-多个开发循环-模型(例如我们使用的模型),自动化是必要的,而不是一种选择。 例如,在任何特定时刻,我们可能会在紧急错误修复循环(需要尽快进行生产)和标准错误修复循环(可能要等待两个或两个以上)的旁边存在一个标准功能开发循环。此外。 由于各种原因,每个循环将在将来的某个未知时间进行部署。 每个循环通常都带有一个专用分支,然后开发人员会创建短暂的分支,最终将这些分支合并到该专用分支中。

Pretty complex with multiple teams working towards one common regular deployment, isn’t it? Well, it was, until we decided to solve it with an internal tool — The Automerger. It’s a tool that we created during our “branching story” at the EcoVadis Engineering team and in this article, we’ll show you how it helped us…

由多个团队共同完成一个常规的常规部署非常复杂,不是吗? 好吧,直到我们决定使用内部工具“自动合并”解决它为止。 这是我们在EcoVadis工程团队的“分支故事”中创建的工具,在本文中,我们将向您展示它如何帮助我们……

问题 (Problem)

Let’s examine the situation where the development team has three active branches. A common situation with GitFlow branching pattern:

让我们研究一下开发团队有三个活跃分支的情况。 GitFlow分支模式的常见情况:

Image for post

Master branch is representing the current production state.

分支代表当前的生产状态。

The first working branch is Develop. Develop branch contains all the features that are going to be included in a release in the future; moreover, it doesn’t matter when this release happens.

第一个工作分支是Develop发展 分支包含将来发行版中将包含的所有功能; 此外,此版本何时发布都无关紧要。

The second branch, let’s name it Release-8 is going to be deployed to production very soon. No feature development on this branch is happening at this precise moment. Release-8 is currently in the stabilization phase. Let’s think about it as it is in QA hands. Only bug fixes can go there.

第二个分支,我们命名为Release-8 ,它将很快部署到生产中。 此时此分支没有功能开发。 Release-8当前处于稳定阶段。 让我们考虑一下质量检查手中的情况。 只有错误修复可以去那里。

What’s the dilemma? Let’s imagine a situation where one bugfix goes to Release-8 and does not go to develop. Next Release-9 would reintroduce bugs. A situation like that is unquestionably not desired. Yep, you can ask developers to put changes into two branches at the time, once for the Release-8 and second time to develop. Sure, but a human tends to make mistakes, someone can forget. Develop can be a way ahead of Release-8, which makes it hard to put the same change on two branches. A developer can simply omit this duty. That’s life, what can we do about it? Automate it.

什么是困境? 让我们想象一种情况,其中一个错误修复发布Release 8 ,而不再开发。 下一版本9将重新引入错误。 毫无疑问,这种情况是不希望的。 是的,您可以要求开发人员一次将更改放入两个分支,一次用于Release-8 ,第二次进行开发。 当然,但是一个人容易犯错误,有人会忘记。 Develop可以比Release-8领先,这使得很难在两个分支上进行相同的更改。 开发人员可以简单地忽略此职责。 那就是生命,我们该怎么办? 自动化它。

This is where the Automerger tool comes in place. It is intended to detect branches assigned to the development loops and perform in-place merges between them systematically. Whenever it finds any conflicts during the merge, it notifies an appropriate team that user action is required.

这是自动合并工具到位的地方。 它旨在检测分配给开发循环的分支,并系统地在它们之间执行就地合并。 只要在合并过程中发现任何冲突,它就会通知适当的团队需要用户采取行动。

独立工具 (Standalone tools)

Automerger如何帮助我的团队/团队? (How can Automerger help my team/teams?)

If your teams are in a position where you perform merges that are required by the branching strategy, or there are technical merges that have to be made periodically for any reason — Automerger comes into play. It takes the duty of your team and allows them to concentrate on delivering value to your company rather than completing boring merges. Do not lose time on tasks that can be done automatically.

如果您的团队处于执行分支策略所需的合并的位置,或者由于某种原因必须定期进行技术合并-自动合并将发挥作用。 它承担了团队的职责,并允许他们专注于为公司创造价值,而不是完成无聊的合并。 不要在可以自动完成的任务上浪费时间。

什么是自动合并? (What is Automerger?)

Technically, it is a tool built with Cake. Sources can be found here: GitHub. It makes extensive use of the git command line output. Please feel free to fork the repo (or contribute to it via pull requests) and put your changes there. We will be delighted if you notify us ;)

从技术上讲,它是使用Cake构建的工具。 可以在这里找到源: GitHub 。 它广泛使用了git命令行输出。 请随时派生仓库(或通过拉取请求贡献仓库)并将更改放在此处。 如果您通知我们,我们将非常高兴;)

流。 内部如何运作? (Flow. How does it work inside?)

One of the goals of the tool was to keep it as simple as possible. As an input, it takes two required parameters: source branch, target branch and executes the following steps:

该工具的目标之一是使其尽可能简单。 作为输入,它需要两个必需的参数:源分支,目标分支,并执行以下步骤:

  1. Create a temporary branch based on the source branch. The naming of the temporary branch can also be customized if you need.

    基于源分支创建一个临时分支。 如果需要,还可以自定义临时分支的命名。
  2. Merge to the temporary branch target branch.

    合并到临时分支目标分支。
  3. Parse output, confirm whether there are conflicts or not. If so, notify a particular developer to perform a manual merge procedure.

    解析输出,确认是否存在冲突。 如果是这样,请通知特定的开发人员以执行手动合并过程。
  4. Usually, when there is no conflict on the temporary branch, the script performs Cake task “Default”.

    通常,当临时分支上没有冲突时,脚本将执行Cake任务“ Default”

  5. After properly finish the default target, the script merges temp branch to the destination branch.

    正确完成默认目标后,脚本会将temp分支合并到目标分支。
  6. Push changes to the origin.

    将更改推送到原点。

There is one external dependency inside the specified flow. I mean point (4) with the execution of the “Default” target from the main Cake script. Its purpose is to validate the codebase after the merge. It is up to the script to run a build of the whole solution with unit test execution. It is necessary to validate the condition of the codebase at this point.

在指定的流中有一个外部依赖项。 我的意思是第(4)点是从主Cake脚本执行“默认”目标。 其目的是在合并后验证代码库。 由脚本来运行带有单元测试执行的整个解决方案的构建。 这时有必要验证代码库的条件。

The Automerger is supposed to be an automatic tool, as few manual interventions as possible should be needed. Based on that assumption, there is a need for an automatic way to validate the correctness of performed merge-operations. The easiest way to achieve that is by building all projects with as many checks as possible. For example unit, integration or acceptance test. Of course, including more checks does not come for free — it increases the execution time significantly. Bear in mind that usually there are more than one of the executions of the Automerge script at a particular time.

自动合并应该是一种自动工具,因此需要的人工干预最少。 基于该假设,需要一种自动方法来验证所执行的合并操作的正确性。 最简单的方法是通过所有项目进行尽可能多的检查。 例如单元测试,集成测试或验收测试。 当然,包含更多检查不是免费的-它会大大增加执行时间。 请记住,通常在特定时间有不止一个Automerge脚本执行。

该平台,只有.Net? (The platform, only .Net?)

Image for post
The Automerger execution log
自动合并执行日志

The Cake is Roslyn and .Net based. It allows you to write scripts using plain C#.

蛋糕基于Roslyn和.Net。 它允许您使用纯C#编写脚本。

Automerger is C# based. But, this visible coupling to the .Net platform is not a blocker. The Automerger works with different platforms. At this moment at our place, the EcoVadis team has seven separate instances of the Automerger running simultaneously. Only two of them are for .Net based projects. The following five projects are JavaScript based frontend projects using React. It was achieved thanks to adjusting one of the steps performed by the script. Following point (4) from the flow described above, it is possible to adjust the script to perform a proper build of code based using JavaScript or Java, for example. You simply need to provide a command-line external tool that performs validation of code state.

自动合并基于C#。 但是,与.Net平台的这种可见耦合并不是一个障碍。 自动合并器可在不同平台上使用。 在我们这一刻,EcoVadis团队有七个同时运行的Automerger独立实例。 其中只有两个用于基于.Net的项目。 以下五个项目是使用React的基于JavaScript的前端项目。 这是由于调整了脚本执行的步骤之一而实现的。 根据上述流程的第(4)点,例如可以调整脚本以使用JavaScript或Java进行适当的代码构建。 您只需要提供一个命令行外部工具即可执行代码状态验证。

约定优于配置 (Convention over configuration)

Usually, the branching model defines the naming of branches that are created during development. The Automerger takes advantage of such conventions. Based on branch names, the Automerger knows which branches should be merged to which. Having branches named:

通常,分支模型定义在开发过程中创建的分支的命名。 自动合并利用了这些约定。 根据分支名称,自动合并器知道哪些分支应合并到哪个分支。 分支机构名为:

master, develop, release/release-8.0, release/release-8.1, release /release-9.0

掌握,开发,发行/发行版8.0,发行/发行版8.1,发行/发行版9.0

Image for post

The script parses names and sort branches in order:

该脚本按顺序解析名称并排序分支:

  1. Master

  2. Release-9.0

    9.0版

  3. Release-8.1

    版本8.1

  4. Release-8.0

    8.0版

  5. Develop

    发展

Depending on how many active release branches there are, it performs merging as follows Master -> Release-9.0 -> Develop. Introducing a new branch that follows the convention is automatically picked up by the tool. No additional configuration needed.

根据存在的活动发行分支的数量,它按以下步骤进行合并: Master- > Release-9.0- > Develop 。 该工具会自动引入遵循约定的新分支。 无需其他配置。

局限性 (Limitations)

Automerger requires a way to check whether changes on one branch are already present on the target branch. If you decide to perform proper merges from one branch to another, rather than doing cherrypicks, you are in a safe place. Git checks it for you, based on the hash of each commit. Automerger is based on this assumption. At this moment, if your team decides to go with cherrypicks and moving changes between branches with creating new commits each time. Automerger gets confused. It notifies you about the vast changes list and probably many conflicts.

自动合并需要一种方法来检查目标分支上是否已经存在一个分支上的更改。 如果您决定从一个分支执行正确的合并到另一个分支,而不是执行樱桃签,那么您将处于安全的位置。 Git根据每次提交的哈希值为您检查它。 自动合并基于此假设。 目前,如果您的团队决定采用“樱桃挑剔”,并且每次创建新的提交都在分支之间移动更改。 自动合并变得困惑。 它通知您有关巨大的更改列表以及可能存在的许多冲突。

可扩展性和可维护性 (Scalability and maintainability)

The Automerger is present in the form of a script. Logically it is present as at least two separate files. These files need to be in a repository where the script is working. This limitation makes the Automerger a bit tricky to scale out and maintain in the longer run. Each time you create a new repository, it needs to be provided with those scripts inside. Each repo has a new instance of Automerger files. As a solution for this, you can create a template for your empty repository containing the Automerger files along with .gitignore file.

自动合并以脚本形式存在。 从逻辑上讲,它作为至少两个单独的文件存在。 这些文件必须位于脚本运行所在的存储库中。 这种局限性使得Automerger在扩展和维持长期运行方面有些棘手。 每次创建新的存储库时,都需要在其中提供这些脚本。 每个存储库都有一个新的Automerger文件实例。 作为解决方案,您可以为空存储库创建一个模板,其中包含Automerger文件以及.gitignore文件。

聚在一起 (Getting all together)

CI / CD中的自动合并 (Automerger in CI/CD)

The Automerger achieves full usability when it goes hand to hand with tools like Azure DevOps or any other CI/CD platform. CI part provides the possibility to schedule a build that executes the Automerger periodically. You can find out of the box build definition for VSTS here: GitHub.

当与Azure DevOps或任何其他CI / CD平台等工具结合使用时,自动合并器可实现完全的可用性。 CI部分提供了安排计划定期执行自动合并的构建的可能性。 您可以在此处找到 VSTS的现成版本定义: GitHub 。

Image for post
The Automerger build configuration
自动合并构建配置

统计资料 (Stats)

After two years of using the tool, it proved itself working and merging in around 90%. What it means, loosely nine of ten times the Automerger do merge changes between branches without any merging conflict. Nine of ten times, a set of changes is propagated between branches without a single person knowing. The 10% scenario is a case when the Automerger informs the author about needed intervention.

使用该工具两年后,它证明自己可以正常工作并合并到90%。 这意味着,自动合并的十次中有九次确实合并了分支之间的更改,而没有任何合并冲突。 十次中的九次,一组更改在分支机构之间传播,而没有一个人知道。 当自动合并功能通知作者需要的干预时,情况为10%。

We do not keep the data from the beginning of time due to data retention policy. Last 7 days of data at the moment when the article was published looks like:

由于数据保留政策,我们不会从开始就保留数据。 该文章发布时的最后7天的数据如下:

Image for post

常问问题 (FAQ)

运行Automerger是否有最低要求? (Is there a minimum requirement to run Automerger?)

As long as you fulfil the requirements for the Cake build tool, you are fine. Minimal requirements do not come directly from Automerger itself. Requirements are more related to your building process itself. Please be sure you have put all required external dependencies in place to have a proper build process. For example, you have a smoke test calling Elasticsearch as a part of artefact verification, please be sure Elasticsearch instances are available from the build machine, etc..

只要满足Cake制作工具的要求 ,就可以了。 最低要求并非直接来自于Automerger本身。 需求与您的构建过程本身更为相关。 请确保已放置所有必需的外部依赖项以具有正确的构建过程。 例如,您有一个抽烟测试,它在工件验证中调用了Elasticsearch,请确保从构建机器等获得了Elasticsearch实例。

您多久运行一次脚本? (How often do you run the script?)

After a time, we have learned that as often as possible synchronization is crucial. Postponing merging of changes increases the risk of a potential merge conflict significantly, also increases the difference between the two branches. It is generally not recommended to perform the huge merge, way better is to have them more and smaller. We have decided to perform script runs every ~3–4 hours per working day. Usually it is 5am, 8am, 12pm, 3pm, 6pm. Hours were chosen empirically. We have found out it is a good balance between the execution time and development speed of the engineering teams and build agent occupation. You may want to pick your hours yourself. I suggest running it as often as possible.

一段时间之后,我们了解到,尽可能频繁地进行同步至关重要。 推迟变更的合并会显着增加潜在合并冲突的风险,也增加两个分支之间的差异。 通常不建议执行大型合并,更好的方法是使它们越来越小。 我们决定在每个工作日每3-4小时执行一次脚本运行。 通常是凌晨5点,8点,12点,3点,6点。 时间是根据经验选择的。 我们发现,在工程团队的执行时间和开发速度与建立代理人之间取得了很好的平衡。 您可能要自己选择时间。 我建议尽可能经常运行它。

您使用Slack通知,我们的团队正在使用邮件通讯。 (You use Slack notifications, our team is using mail communication.)

No problem with that, please feel free to make code changes with the mailing mechanism and share a pull request. I will be more than happy to cooperate. Please, also find that cake building tool provides a wide array of out-of-the-box plugins that can be used. You can find add-ins here: https://cakebuild.net/addins/

没问题,请随时通过邮件机制更改代码并共享请求请求。 我将很高兴合作。 还请您发现,蛋糕构建工具提供了许多可以使用的即用型插件。 您可以在这里找到加载项: https : //cakebuild.net/addins/

我们有一个以上的团队,其中20个团队各有10个开发人员。 (We have more than one team, 20 teams with 10 devs each.)

No problem. Please find out file: .automergeConfig. It contains a simple mapping between developers and the team. Whenever the Automerger finds a conflict during merge, it looks for the last person that edited a particular file. Then, the script finds the right team and notifies the team — no need to notify everyone in your IT department.

没问题。 请找出文件:.automergeConfig。 它包含开发人员和团队之间的简单映射。 每当自动合并器在合并期间发现冲突时,它都会寻找编辑特定文件的最后一个人。 然后,脚本可以找到合适的团队并通知该团队-无需通知您IT部门的所有人。

可以,但是我想通知所有人。 (Sure, but I want to notify everyone.)

The default configuration will do it then ;)

然后,默认配置将完成;)

随着时间的推移,自动合并会产生很多分支吗? (Does The Automerger produce quite a several branches over time?)

It depends on configuration. The script can go both ways. It can leave the branch with conflict for your team to follow later. Depending on how often you execute the script, it creates a new temporary branch each time. It is not recommended, we have found out that it is better to allow The Automerger to clean after itself. It allows work to be more asynchronous. The developer does not need to take care of conflicts before the next Automerger’s run. On the other hand, the Automerger’s build can run as much as it wants to not be blocked by the developer.

这取决于配置。 该脚本可以双向执行。 它可能会使分支出现冲突,以供您的团队以后使用。 根据您执行脚本的频率,脚本每次都会创建一个新的临时分支。 不建议这样做,我们发现最好让The Automerger自行清理。 它使工作更加异步。 在下一次Automerger运行之前,开发人员无需处理冲突。 另一方面,Automerger的构建可以在不被开发人员阻止的情况下运行。

摘要 (Summary)

Automation is crucial. Do not let your team or yourself find a comfortable place, repeating the same task day by day. Building a tool for task automation allows you to move on and focus on tasks that bring company value. The Automerger tackles one of these day-by-day struggles. In this particular case, it is all about repeatable merges.

自动化至关重要。 不要让您的团队或您自己找一个舒适的地方,每天重复同一任务。 通过构建用于任务自动化的工具,您可以继续进行并专注于带来公司价值的任务。 汽车合并解决了这些日常斗争之一。 在这种特殊情况下,所有内容都与可重复合并有关。

翻译自: https://medium.com/ecovadis-engineering/the-automerger-216d1b060396

合并报表excel自动模板


http://www.niftyadmin.cn/n/1468587.html

相关文章

工业互联网(十一)——Agilent函数发生器编程(Agilent IO Suite使用)

实验室买了个函数发生器,要集成到自己的设备上,又想从自己编写的PC Windows软件上直接控制,不用手动操作。像这种标准测试控制仪器都是提供PC接口的,在查阅了相关资料以后,完美解决问题,记下相关知识以供有…

comparable的使用场景_在大型项目中如何使用Git子模块开发,看完涨姿势了!

点击上方“Java知音”,选择“置顶公众号”技术文章第一时间送达!作者:Maxliujuejin.im/post/5c1c5d305188256a272aa0ec写在前面公司需要开发一个内部系统,要求每个部门都要接入。老板钦点,工期又压得短,于是…

工业互联网(十二)——VISA编程

1.试水VISA编程(1)——对仪器的基本的控制 由于项目所需,要编写上位机控仪器实现所谓一键读取数据的功能。于是乎不得不学习点VISA编程。研究了一阵子手册,明白了发送命令的方式。使用的仪器是安捷伦的DSO-X 3034A。 1、在https…

高德地图 自动计算缩放_您应该了解的无服务器计算中的自动缩放模式

高德地图 自动计算缩放Pioneered by AWS Lambda, serverless computing is the latest paradigm in cloud service. Nowadays, almost all large cloud provider companies are offering Function-as-a-Service (FaaS) products which promise developers the total management…

vb和python内容的区别_vb.net和python区别

vb.net和python区别 Python的应用面相对广,跨平台,而且易于上手,代码条理清晰,人工智能,自动化任务,以及Web应用都算是Python一展所长的地方。 VB算是比较老的编程语言,是编译型语言&#xff0c…

C#学习(十六)——基于C#与OpenCVSharp的图像处理软件

基于OpenCVSharp的图像处理软件: https://blog.csdn.net/you_big_father/article/details/86088531?utm_mediumdistribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-9.nonecase&depth_1-utm_sourcedistribute.pc_relevant.none-task-blog…

ubuntu 运行grpc_在生产中运行grpc服务的挑战

ubuntu 运行grpcThere are several ways to make services communicate, which generally involve a transport layer. Our applications often rely on it to provide several abstractions and features, such as load balancing, retries and high availability.有几种方法可…

OpenCVsharp学习笔记(一)

使用opencvsharp在c# winform的pictureBox上播放视频 https://blog.csdn.net/zanllp/article/details/82263751 c# opencvsharp学习笔记(1) https://blog.csdn.net/zanllp/article/details/79828968 c# opencvsharp学习笔记(2&#xff09…