Ошибка резервного копирования на гугл диск

[VadiKO]

Уже несколько дней на всех серверах завершается ошибкой резервное копирование на гугл диск.

Ранее такие ошибки бывали раз в несколько дней, сейчас же усугубилось полностью.

Так у всех? Что-то изменилось со стороны гугла? Или искать проблему у себя?

Спасибо.

[Besar]

По крайней мере с v.5.241.1 Lite (Debian 10) не наблюдаю, а до этого иногда были ошибки.

[VadiKO]

Хм, у меня два разных сервера, правда CENTOS 7, ISPmanager Lite 5.241.1-2020.03.13_17:13 и на обеих уже несколько дней все заканчивается - Backup root 2020-04-10 Failed to create backup

Подскажите, пожалуйста, куда копать?

[Iliaxxx]

Хм, у меня два разных сервера, правда CENTOS 7, ISPmanager Lite 5.241.1-2020.03.13_17:13 и на обеих уже несколько дней все заканчивается - Backup root 2020-04-10 Failed to create backup

Подскажите, пожалуйста, куда копать?

А что у Вас в логах?

[VadiKO]

В логах сегодня было такое:

Apr 10 03:03:32 [24801:1] rpc [1;36mEXTINFO Query('func=backup2.item')[0m
Apr 10 03:03:32 [24801:1] proc [1;36mEXTINFO Run 'sbin/backup2 --date 2020-04-10 root' pid 25546[0m
Apr 10 03:03:32 [25546:1] libmgr [1;36mEXTINFO Command line argument 'date' add value '2020-04-10'[0m
Apr 10 03:03:32 [25547:1] rpc [1;36mEXTINFO Query('func=backup2.item.info&elid=root&role=')[0m
Apr 10 03:03:32 [25547:1] proc [1;36mEXTINFO Run 'sbin/backup2_cp --server --path var/backup/ispmgr --date 2020-04-10 --count-limit 2:3' pid 25549[0m
Apr 10 03:03:32 [25547:1] proc [1;36mEXTINFO Run 'nice -n 10 ionice -c2 -n7 sbin/isptar --create var/backup/ispmgr/root/2020-04-10/I2020-04-10.root.tgz --slice 100M --root / --save-listing var/backup/ispmgr/root/2020-04-10/2020-04-10.root.tgz --base var/backup/ispmgr//root/2020-04-05/2020-04-05.root.tgz --exclude data/logs/\*.log --exclude data/mod-tmp --exclude data/bin-tmp --execute sbin/backup2_cp\ --put\ --info-file\ var/backup/ispmgr/root/2020-04-10/2020-04-10.root.info.7rf6EC\ --%c\ %p/%f etc usr/local/mgr5/etc usr/local/mgr5/var/userconf' pid 25550[0m
Apr 10 03:03:36 [25547:1] proc [1;36mEXTINFO Process 25550 finished with status 0[0m
Apr 10 03:03:36 [25547:1] proc [1;36mEXTINFO Run '/bin/sh -c sbin/backup2_cp\ --put\ --info-file\ var/backup/ispmgr/root/2020-04-10/2020-04-10.root.info.7rf6EC\ --isolated\ --no-delete\ var/backup/ispmgr/root/2020-04-10/2020-04-10.root.tgz' pid 25554[0m
Apr 10 03:03:39 [25547:1] proc [1;36mEXTINFO Process 25554 finished with status 1[0m
Apr 10 03:03:39 [25547:1] backup [1;31mERROR final backup2_cp call finished with error 1. Output: 
[0m
Apr 10 03:03:39 [25547:1] rpc [1;36mEXTINFO Query('func=backup2.item&finish=3&user=root')[0m
Apr 10 03:03:39 [25547:1] rpc [1;36mEXTINFO Query('func=problems.register&id=root&level=error&info=Failed%20to%20create%20backup&name=backup2')[0m
Apr 10 03:03:40 [25547:1] proc [1;36mEXTINFO Run '/bin/sh -c sbin/backup2_cp\ --delete\ var/backup/ispmgr/root/2020-04-10/2020-04-10.root.info.7rf6EC' pid 25562[0m
Apr 10 03:03:40 [25547:1] proc [1;36mEXTINFO Process 25562 finished with status 0[0m
Apr 10 03:03:40 [25547:1] proc [1;35mWARNING Process 25549 killed. waitpid still need[0m
Apr 10 03:03:40 [24801:1] proc [1;36mEXTINFO Process 25546 finished with status 0[0m
Apr 10 03:03:40 [24801:1] rpc [1;36mEXTINFO Query('func=backup2.item')[0m
Apr 10 03:03:40 [24801:1] backup2 [1;32mINFO Finished[0m

[VadiKO]

Прошла ночь и снова:

New Problem
Во время резервного копирования пользователя root произошла ошибка

В файле backup2.log:

Apr 11 03:00:02 [20788:1] rpc [1;36mEXTINFO Query('func=backup2.item&start=on&date=2020%2D04%2D11')[0m
Apr 11 03:00:02 [20788:1] rpc [1;36mEXTINFO Query('func=backup2.item')[0m
Apr 11 03:00:02 [20788:1] proc [1;36mEXTINFO Run 'sbin/backup2 --date 2020-04-11 admin' pid 20815[0m
Apr 11 03:00:02 [20815:1] libmgr [1;36mEXTINFO Command line argument 'date' add value '2020-04-11'[0m
Apr 11 03:00:02 [20815:1] rpc [1;36mEXTINFO Query('func=user.edit&elid=admin')[0m
Apr 11 03:00:02 [20818:1] rpc [1;36mEXTINFO Query('func=backup2.item.info&elid=admin&role=')[0m
Apr 11 03:00:02 [20818:1] proc [1;36mEXTINFO Run 'sbin/backup2_cp --server --path var/backup/ispmgr --date 2020-04-11 --count-limit 2:3' pid 20820[0m
Apr 11 03:00:03 [20818:1] proc [1;36mEXTINFO Run 'nice -n 10 ionice -c2 -n7 sbin/isptar --create var/backup/ispmgr/admin/2020-04-11/F2020-04-11.admin.tgz --slice 100M --user admin --root /var/www/admin --save-listing var/backup/ispmgr/admin/2020-04-11/2020-04-11.admin.tgz --exclude data/logs/\*.log --exclude data/mod-tmp --exclude data/bin-tmp --execute sbin/backup2_cp\ --put\ --info-file\ var/backup/ispmgr/admin/2020-04-11/2020-04-11.admin.info.n5dZQk\ --%c\ %p/%f --backup-hook .system/ --backup-hook-execute sbin/backup2_system\ --%c\ --user\ admin\ --manager\ ispmgr\ %p/%f .system data' pid 20825[0m
Apr 11 03:04:48 [20818:1] proc [1;36mEXTINFO Process 20825 finished with status 1[0m
Apr 11 03:04:48 [20818:1] backup [1;31mERROR isptar finished with error 1. Output: 
Failed to upload data
[0m
Apr 11 03:04:48 [20818:1] rpc [1;36mEXTINFO Query('func=backup2.item&finish=3&user=admin')[0m
Apr 11 03:04:48 [20818:1] rpc [1;36mEXTINFO Query('func=problems.register&id=admin&level=error&info=Failed%20to%20create%20backup&name=backup2')[0m
Apr 11 03:04:48 [20818:1] proc [1;36mEXTINFO Run '/bin/sh -c sbin/backup2_cp\ --delete\ var/backup/ispmgr/admin/2020-04-11/2020-04-11.admin.info.n5dZQk' pid 21544[0m
Apr 11 03:04:49 [20818:1] proc [1;36mEXTINFO Process 21544 finished with status 0[0m
Apr 11 03:04:49 [20818:1] proc [1;35mWARNING Process 20820 killed. waitpid still need[0m
Apr 11 03:04:49 [20788:1] proc [1;36mEXTINFO Process 20815 finished with status 0[0m
Apr 11 03:04:49 [20788:1] rpc [1;36mEXTINFO Query('func=backup2.item')[0m
Apr 11 03:04:49 [20788:1] proc [1;36mEXTINFO Run 'sbin/backup2 --date 2020-04-11 root' pid 21547[0m
Apr 11 03:04:49 [21547:1] libmgr [1;36mEXTINFO Command line argument 'date' add value '2020-04-11'[0m
Apr 11 03:04:49 [21548:1] rpc [1;36mEXTINFO Query('func=backup2.item.info&elid=root&role=')[0m
Apr 11 03:04:49 [21548:1] proc [1;36mEXTINFO Run 'sbin/backup2_cp --server --path var/backup/ispmgr --date 2020-04-11 --count-limit 2:3' pid 21550[0m
Apr 11 03:04:49 [21548:1] proc [1;36mEXTINFO Run 'nice -n 10 ionice -c2 -n7 sbin/isptar --create var/backup/ispmgr/root/2020-04-11/F2020-04-11.root.tgz --slice 100M --root / --save-listing var/backup/ispmgr/root/2020-04-11/2020-04-11.root.tgz --exclude data/logs/\*.log --exclude data/mod-tmp --exclude data/bin-tmp --execute sbin/backup2_cp\ --put\ --info-file\ var/backup/ispmgr/root/2020-04-11/2020-04-11.root.info.Nu21q3\ --%c\ %p/%f etc usr/local/mgr5/etc usr/local/mgr5/var/userconf' pid 21551[0m
Apr 11 03:05:01 [21548:1] proc [1;36mEXTINFO Process 21551 finished with status 0[0m
Apr 11 03:05:01 [21548:1] proc [1;36mEXTINFO Run '/bin/sh -c sbin/backup2_cp\ --put\ --info-file\ var/backup/ispmgr/root/2020-04-11/2020-04-11.root.info.Nu21q3\ --isolated\ --no-delete\ var/backup/ispmgr/root/2020-04-11/2020-04-11.root.tgz' pid 21556[0m
Apr 11 03:05:08 [21548:1] proc [1;36mEXTINFO Process 21556 finished with status 1[0m
Apr 11 03:05:08 [21548:1] backup [1;31mERROR final backup2_cp call finished with error 1. Output: 
[0m
Apr 11 03:05:08 [21548:1] rpc [1;36mEXTINFO Query('func=backup2.item&finish=3&user=root')[0m
Apr 11 03:05:08 [21548:1] rpc [1;36mEXTINFO Query('func=problems.register&id=root&level=error&info=Failed%20to%20create%20backup&name=backup2')[0m
Apr 11 03:05:08 [21548:1] proc [1;36mEXTINFO Run '/bin/sh -c sbin/backup2_cp\ --delete\ var/backup/ispmgr/root/2020-04-11/2020-04-11.root.info.Nu21q3' pid 21609[0m
Apr 11 03:05:08 [21548:1] proc [1;36mEXTINFO Process 21609 finished with status 0[0m
Apr 11 03:05:09 [21548:1] proc [1;35mWARNING Process 21550 killed. waitpid still need[0m
Apr 11 03:05:09 [20788:1] proc [1;36mEXTINFO Process 21547 finished with status 0[0m
Apr 11 03:05:09 [20788:1] rpc [1;36mEXTINFO Query('func=backup2.item')[0m
Apr 11 03:05:09 [20788:1] backup2 [1;32mINFO Finished[0m

backup2_system.log:

Apr 11 03:00:03 [20826:1] libmgr [1;36mEXTINFO Command line argument 'manager' add value 'ispmgr'[0m
Apr 11 03:00:03 [20826:1] rpc [1;36mEXTINFO Query('func=backup2.db&elid=mysql%2Ef%5Fadminbil&user=adminoneinfo&filename=%2Fvar%2Fwww%2Fadminoneinfo%2F%2Esystem%2Fdb%2Emysql%2Ef%5Fadminbil&command=dump')[0m
Apr 11 03:00:08 [20836:1] libmgr [1;36mEXTINFO Command line argument 'user' add value 'adminoneinfo'[0m
Apr 11 03:00:08 [20836:1] libmgr [1;36mEXTINFO Command line argument 'manager' add value 'ispmgr'[0m
Apr 11 03:00:09 [20837:1] libmgr [1;36mEXTINFO Command line argument 'user' add value 'adminoneinfo'[0m
Apr 11 03:00:09 [20837:1] libmgr [1;36mEXTINFO Command line argument 'manager' add value 'ispmgr'[0m
Apr 11 03:00:09 [20837:1] rpc [1;36mEXTINFO Query('func=backup2.db&elid=mysql%2Eadminbiling&user=adminoneinfo&filename=%2Fvar%2Fwww%2Fadminoneinfo%2F%2Esystem%2Fdb%2Emysql%2Eadminbiling&command=dump')[0m
Apr 11 03:00:48 [20852:1] libmgr [1;36mEXTINFO Command line argument 'user' add value 'adminoneinfo'[0m
Apr 11 03:00:48 [20852:1] libmgr [1;36mEXTINFO Command line argument 'manager' add value 'ispmgr'[0m
Apr 11 03:00:48 [20853:1] libmgr [1;36mEXTINFO Command line argument 'user' add value 'adminoneinfo'[0m
Apr 11 03:00:48 [20853:1] libmgr [1;36mEXTINFO Command line argument 'manager' add value 'ispmgr'[0m
Apr 11 03:00:48 [20853:1] rpc [1;36mEXTINFO Query('func=backup2.metadata&elid=xml&user=adminoneinfo&filename=%2Fvar%2Fwww%2Fadminoneinfo%2F%2Esystem%2Fmetadata%2Exml&command=dump')[0m
Apr 11 03:01:05 [21461:1] libmgr [1;36mEXTINFO Command line argument 'user' add value 'adminoneinfo'[0m
Apr 11 03:01:05 [21461:1] libmgr [1;36mEXTINFO Command line argument 'manager' add value 'ispmgr'[0m

[VadiKO]

backup2_cp.log:

Apr 11 03:04:49 [21544:1] backtrace [1;36mEXTINFO ??? (/lib64/libc.so.6 + 0xf5) [*0x4015f7][0m
Apr 11 03:04:49 [21544:1] backtrace [1;36mEXTINFO ??? (sbin/backup2_cp + 0x0) [*(nil)][0m
Apr 11 03:04:49 [21544:1] libmgr [1;31mERROR Error: Type: 'rpc' Object: 'query' Value: 'query: Failed writing received data to disk/application'[0m
Apr 11 03:04:49 [21550:1] libmgr [1;36mEXTINFO Command line argument 'path' add value 'var/backup/ispmgr'[0m
Apr 11 03:04:49 [21550:1] libmgr [1;36mEXTINFO Command line argument 'date' add value '2020-04-11'[0m
Apr 11 03:04:49 [21550:1] libmgr [1;36mEXTINFO Command line argument 'count-limit' add value '2:3'[0m
Apr 11 03:04:49 [21550:1] main [1;32mINFO Size control server is ready. Date: 2020-04-11[0m
Apr 11 03:04:59 [21553:1] libmgr [1;36mEXTINFO Command line argument 'info-file' add value 'var/backup/ispmgr/root/2020-04-11/2020-04-11.root.info.Nu21q3'[0m
Apr 11 03:04:59 [21553:1] main [1;32mINFO Put /usr/local/mgr5/var/backup/ispmgr/root/2020-04-11/F2020-04-11.root.tgz[0m
Apr 11 03:04:59 [20824:1] main [1;36mEXTINFO PUT result: 'OK root/2020-04-11/F2020-04-11.root.tgz.aes 17149509 52c9c36ef219f5b44bfbf9397b62be16e1dbda991dcd2090de41a935d9c460e2'[0m
Apr 11 03:05:01 [20824:3] proc [1;36mEXTINFO Run '/bin/sh -c openssl\ enc\ -e\ -pass\ stdin\ -aes-256-cbc\ -in\ /usr/local/mgr5/var/backup/ispmgr/root/2020-04-11/F2020-04-11.root.tgz' pid 21557[0m
Apr 11 03:05:01 [21556:1] libmgr [1;36mEXTINFO Command line argument 'info-file' add value 'var/backup/ispmgr/root/2020-04-11/2020-04-11.root.info.Nu21q3'[0m
Apr 11 03:05:01 [21556:1] main [1;32mINFO Put /usr/local/mgr5/var/backup/ispmgr/root/2020-04-11/2020-04-11.root.tgz[0m
Apr 11 03:05:01 [20824:1] main [1;36mEXTINFO PUT result: 'OK root/2020-04-11/2020-04-11.root.tgz.aes 42870 52c9c36ef219f5b44bfbf9397b62be16e1dbda991dcd2090de41a935d9c460e2'[0m
Apr 11 03:05:07 [20824:3] proc [1;36mEXTINFO Process 21557 finished with status 0[0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO mgr_rpc::Fail::Fail(std::string const&, CURLcode) (lib/libmgr.so.5.241.0 + 0xba) [*0x7fea0a339d98][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO mgr_rpc::HttpQuery::Implementation::Perform(mgr_rpc::HttpQuery&, mgr_rpc::HttpQuery::Implementation::method_t, std::string const&, std::string const&, std::ostream&) (lib/libmgr.so.5.241.0 + 0x9c8) [*0x7fea0a335fd2][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO mgr_rpc::HttpQuery::Get(std::string const&, std::ostream&) (lib/libmgr.so.5.241.0 + 0x62) [*0x7fea0a3b4712][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO mgr_rpc::GoogleDrive::GetName(std::string const&, std::string const&) (lib/libmgr.so.5.241.0 + 0x772) [*0x7fea0a3b8349][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO mgr_rpc::GoogleDrive::MkPath(std::string const&) (lib/libmgr.so.5.241.0 + 0x159) [*0x7fea0a6468fb][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO backup2::Storage::MkDir(std::string const&) (lib/libbackup2.so + 0x24b) [*0x7fea0a897e2e][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO Upload::Upload(NodeSide*, std::string const&, std::string const&) (libexec/backup2_cp.so + 0xbe) [*0x7fea0a89b12d][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO NodeSide::operator()(ResHandle) (libexec/backup2_cp.so + 0x75d) [*0x7fea0a89f701][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO backup2::Expect::Worker<NodeSide>::Do(ResHandle) (libexec/backup2_cp.so + 0x51) [*0x7fea0a664704][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO backup2::Expect::Run(int) (lib/libbackup2.so + 0x1c4) [*0x7fea0a89a508][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO CommandClient() (libexec/backup2_cp.so + 0x368) [*0x7fea0a894b35][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO ??? (libexec/backup2_cp.so + 0x15a5) [*0x401359][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO ??? (/usr/local/mgr5/sbin/backup2_cp + 0x0) [*0x7fea0aadd505][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO ??? (/lib64/libc.so.6 + 0xf5) [*0x4015f7][0m
Apr 11 03:05:08 [20824:1] backtrace [1;36mEXTINFO ??? (/usr/local/mgr5/sbin/backup2_cp + 0x0) [*(nil)][0m
Apr 11 03:05:08 [20824:1] libmgr [1;31mERROR Error: Type: 'rpc' Object: 'query' Value: 'query: Failed writing received data to disk/application'[0m
Apr 11 03:05:08 [20824:1] main [1;31mERROR Type: 'rpc' Object: 'query' Value: 'query: Failed writing received data to disk/application'[0m
Apr 11 03:05:08 [21609:1] main [1;32mINFO Delete /usr/local/mgr5/var/backup/ispmgr/root/2020-04-11/2020-04-11.root.info.Nu21q3[0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO mgr_rpc::Fail::Fail(std::string const&, CURLcode) (lib/libmgr.so.5.241.0 + 0xba) [*0x7f22b9864d98][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO mgr_rpc::HttpQuery::Implementation::Perform(mgr_rpc::HttpQuery&, mgr_rpc::HttpQuery::Implementation::method_t, std::string const&, std::string const&, std::ostream&) (lib/libmgr.so.5.241.0 + 0x9c8) [*0x7f22b9860fd2][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO mgr_rpc::HttpQuery::Get(std::string const&, std::ostream&) (lib/libmgr.so.5.241.0 + 0x62) [*0x7f22b98df2c1][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO mgr_rpc::GoogleDrive::GetName(std::string const&, std::string const&) (lib/libmgr.so.5.241.0 + 0x321) [*0x7f22b98e0605][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO mgr_rpc::GoogleDrive::GetID(std::string const&) (lib/libmgr.so.5.241.0 + 0xb5) [*0x7f22b98e2bf3][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO mgr_rpc::GoogleDrive::RmDir(std::string const&) (lib/libmgr.so.5.241.0 + 0x273) [*0x7f22b9b79bdf][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO backup2::StorageTemplate<mgr_rpc::GoogleDrive>::RmDir(std::string const&) (lib/libbackup2.so + 0xdf) [*0x7f22b9dd4865][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO Delete(std::shared_ptr<backup2::Storage>, std::string const&, bool) (libexec/backup2_cp.so + 0x5f5) [*0x7f22b9dd6438][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO DeleteWithDepends(std::shared_ptr<backup2::Storage>, std::string const&, bool) (libexec/backup2_cp.so + 0x1d8) [*0x7f22b9dd87e3][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO CommandDelete(std::shared_ptr<mgr_args::Result>) (libexec/backup2_cp.so + 0x193) [*0x7f22b9dbfc37][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO ??? (libexec/backup2_cp.so + 0x16a7) [*0x401359][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO ??? (sbin/backup2_cp + 0x0) [*0x7f22ba008505][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO ??? (/lib64/libc.so.6 + 0xf5) [*0x4015f7][0m
Apr 11 03:05:08 [21609:1] backtrace [1;36mEXTINFO ??? (sbin/backup2_cp + 0x0) [*(nil)][0m
Apr 11 03:05:08 [21609:1] libmgr [1;31mERROR Error: Type: 'rpc' Object: 'query' Value: 'query: Failed writing received data to disk/application'[0m
Apr 11 03:05:09 [20822:1] proc [1;36mEXTINFO Process 20824 finished with status 0[0m
Apr 11 03:05:11 [20822:1] log INFO Finished

[VadiKO]

В общем это проблема на стороне панели, поддержка ответила:

У нас есть задача на исправление этой ошибки, в данный момент ошибка еще не исправлена.
Пожалуйста следите за нашим changelog чтобы быть в курсе всех изменений: https://www.ispsystem.ru/software/ispmanager/changelog