三、大数据平台

M 公司对大数据的管理、开发与应用有着自身独特的需求,必须建设适合自身业务需求[br]的大数据平台来满足业务需求,并能支撑业务的稳定、快速发展的需要。
[b] (一)技术架构[br][br][/b] M 公司从自身的业务需求出发,充分借鉴了其他企业在大数据发展和应用方面的经验,形成了如[color=#0000ff]附图 14 [/color]所示的技术架构。[br][br][br][br][br][br][br][br][br] 附图 14 M 公司的大数据技术架构
从图中可以看出,M 公司的大数据架构主要包括数据采集、数据存储、数据管理、数据分析、算法以及数据可视化等组件,大部分组件均采用了开源技术,并对一些核心组件进行了深加工、优化和自定义。数据采集采用的是 JavaScribe 技术,每台机器都有一个代理软件把数据收集起来。存储部分以 HBase 技术为主,取得了较好的效果。数据管理采用 Kerberos进行认证,准确性和效率都得到了保证。数据分析采用了 MapReduce、Spark、Storm、Hive、Impala、Druid 和 ES 等较为成熟的技术。算法包括机器学习和深度学习等,选用 Tensorflow深度学习平台和 Kubernetes 软件进行一些资源调度和管理,以支持 M 公司的广告、金融、相册和搜索场景等业务。数据可视化包括两条线:一条是通过 Kafka→Druid→数据可视化显示,另一条是完整数据落盘到 HDFS。可视化的方式主要以各种报表为主。
[b] (二)M 数据工场[br][/b][br] 承担 M 公司大数据业务发展的一个重要部门是 M 数据工场,这是一个为全公司业务各[br]团队及 M 生态链企业提供数据采集、计算、存储等基础能力以及机器学习、挖掘工具和方法任务的专门部门,是 M 公司大数据技术开发和资源利用的主力军。数据工场提供支撑 M 公司大数据业务运营的基础性平台,M公司希望通过这一平台能将数据开放给企业的各个部门,促进不同部门之间数据的相互利用,并且希望能加强对数据的权限管理,杜绝数据的不当利用。M 数据工场的总体架构如附图 15 所示。[br][img]data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAkkAAAEBCAYAAABsYzGoAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEnQAABJ0Ad5mH3gAAE3BSURBVHhe7Z0JvFVT+8dX3kJkCpF5eCmvIfNMb2VIkogGEV5UNIm/MXMhGTKWoYhQEVFkKpSUMalMCZmSZo2Izv98n/bKvru9zz3n3lv3ds/v+/mczznrOWvtc+49e631W8/zrL0rpNI4IYQQQghRgHWCZyGEEEIIEUIiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiMF2t7Vo0cINGjTIsdGtQoUKwVvOylASNl+GbNqGyxC1JbWDXI/v64C3x9kgqW1J2rJtB96ey/EhG1u4DFFbcY8P3ubLkE3bcBmitqR2kOvxfR3w9jgbJLUtSVtcHfD2OBsktS2qLVyGXI4P3p6pXrQOZPMZ4TJEbUntINfj+zrg7XE2SGpbkra4OuDtcTZIaltUW7gMuRwfvD1TvaLawmWI2op7fPA2X4Zs2uZ6fMjFFlcHvD3OBklti2oLlyGX44O3Z6pXVBvlZs2auQEDBqzwJCGQhBBCCCHEP7rIPEnrrLOOq1ixonvnnXfMKIQQQgiRjxx11FHur7/+csuXLy8okv7888+gihBCCCFE/rHuuuuuFElK3BZCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEECXKSy+9VKT7QA4bNswNGDAgKGXP6NGj3d9//x2U/mHSpEnus88+C0pCCJE7usGtEKLY/PbbbyZIDj/8cNejRw9XuXJl16FDh+Ddwlm2bJlr27atu+OOO9xmm23m3nvvPVetWjW3yy67BDWSefTRR00QVa9e3S1evNitv/76Np699tpr7oQTTnCXXnppUFMIIQonfINbiSQhRLHBk3PEEUeYuOnXr5/beuutXf369YN3C+fxxx9377//vtt///3dwoUL3cMPP+xeeOEFV6NGjaBGMoikevXque23394xlnkQaT179rSxTQghsiUskhRuE0IUm3/9619ujz32CErO7bPPPsGrwhk7dqyrUqWK22677VyrVq1cet3mevXqlZVAgu+++86NGTPGPfnkk+7BBx80YXT33Xe7jTfe2N4TQoiiIpEkhCgyb775ZvDKuVq1agWvnNtggw3seeDAgRmFyrhx49zvv//umjRp4s4880zXsmVLE0l16tQJahQOIbYDDzzQnXTSSe6cc85xnTt3dhdffLG7+eab3W677RbUEkKI3JFIEkIUmUWLFpnISYKw2S+//BKUVgVvT926de310KFDTSxhQ1whlrKBUB9eqJkzZ7qpU6dauA9vUrt27dyoUaOCWkIIkTsSSUKIInPUUUe5rl27BqUV/Pzzz/aMWGnUqJElcyex5557uhkzZlh4DO8RnqULLrjA8ovat2/vfvzxx6BmPOQMVK1a1cJ9eKwQbY0bN3atW7d2DzzwgKtdu3ZQUwghckciSQhRZNiJ9uWXX7ovvvgisDgTNvPmzTN7p06dAuuqLF261A0fPtx9+umnFh7DI0QuE+Ezdssdf/zxFoIjjMalARBTUebOnWsiiR1tJIofeuih9prvwOcj1G699Vb7PkIIkSva3SaEKBZ9+vRxG264ofv1119N7LCVn91t5AZtsskmQa2CjB8/3k2fPt12sz3//PMman766ScLm5FHNGvWLEsEb9CggYkpcpwqVaoUtP4HkrUrVKjgJk6caJ//xx9/2ANPEvYdd9zRLg3AJQnYsSKEEIWhSwAIIUoMQlyEux566KGVIunCCy8skMidDV26dLFka+BYTZs2NU9VJvASXXXVVfYaccTnYps2bZo75JBDzC6EELmgSwAIIUoMtu/jqQmDBycX8AThVfLMnz/fbbTRRkEpHrxNeJ489913n4msrbbaym2zzTbu6quv1sJPCFEsJJKEECUCOUNcdZv8Hy4M2bdvX/MIkYj9zTffBLVWhR1phNxI8mY3Glff5liFXQRy8ODBrlmzZuZB6t+/vwm1V1991S5M+eGHH67cObdkyZKghRBC5IbCbUKInCG5mtAa+UO4pvHchENjNWvWdJtuumlQcm699daLzQnCgzR58mTXvHlzu1o24qhFixbu3HPPtQtLJsFtSBBjRx99tH2Xbt26ucMOO8x2s5Ef5QUWIo1rJ/H5QgiRDcpJEkKUOlOmTDHvD9v9wyCAuNp2nKjysN1/5513DkpCCFFySCQJIYQQQsQQFknKSRJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYVt677V//+pcbPHhwYBZCCCGEyD9OO+009/fffxe8wS2kX7oKFSrYa6AMJWHzZcimbbgMUVtSO8j1+L4OeHucDZLalqQt23bg7bkcH7KxhcsQtRX3+OBtvgzZtA2XIWpLage5Ht/XAW+Ps0FS25K0xdUBb4+zQVLbotrCZcjl+ODtmepF60A2nxEuQ9SW1A5yPb6vA94eZ4OktiVpi6sD3h5ng6S2RbWFy5DL8cHbM9Urqi1chqituMcHb/NlyKZtrseHXGxxdcDb42yQ1LaotnAZcjk+eHumekW1+XIBkcSjUaNGJfpB4G2+DNm0DZchaktqB7ke39cBb4+zQVLbkrTNnj3bjRkzxu5E3KBBA7PHtQNvL+5nQtQWLkPUVtzjg7f5MmTTNlyGqC2pHeR6fF8HvD3OBkltS9IWVwe8Pc4GSW2LaguXIZfjg7dnqhetA9l8RrgMUVtSO8j1+L4OeHucDZLalqQtrg54e5wNktoW1RYuQy7HHzJkiEU3Tj755Iz1imoLlyFqK+7xwdt8GbJpm+vxIRdbXB3w9jgbJLUtqi1chlyOD96eqV5RbUOHDjWBVEAkVaxY0f35559BNVGajBgxwh133HFu0003dXPnzg2sQghR/mGSYk5ikfjHH38EViHWHJx7f/31l4kkJW4LIYQQQsQgkSSEEEIIEYNEkhBCCCFEDBJJQgghhBAxSCQJIYQQQsRgIontb+EtcKJ08b+HfhMhRL6h8U+UNuHzT54kIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAihtUmkqZMmRK8ygyX/v7222+D0prH37uluCxatCh4JUT2/PTTT27JkiVBqXA+++yzArdq+P777822ppk+fXrwKjcWLlwYvFrBd999Z2OAEPlILud/tnNqJqL9b+rUqSU2B5ZXSkQkcc+3Vq1arXwN9957r5s2bZq9zgT3jLvwwgtXths1apQ9A7aRI0e6zz//PLAUjZ9//jl4tSqTJ092Dz74YFBybvz48cGrFTABcf+Wwnj77bfd6NGj3bvvvusee+yxwCrEP3CeRc8lbuJ5/vnnB6XCefTRRwuIIvpHLoPnW2+9ZcIsSi6iZ86cOa5NmzZBKTeaN29e4B6R999/v/vggw+CUvbwHaK88sorOQlOIdY0t99+u/v4449X9oFly5a5K664wl4XBu0efvhhex2eJ+HXX391/fv3D0rx/P33365Fixb27OnevXuR5te4/jds2LACfbu8UCyRhPfk5ZdftpvB7b333m7p0qXu//7v/+w9JoNtt93WXhfGoYceaseAa665xvXs2dP16dPH3XPPPe61115zM2fOtPfCoH75sXnwgyFOeLzwwgvugQcesMdtt91mExADcdwxgO+9ePHilSfKM8884+6+++6V3+Hiiy92M2bMsPcyMXz4cLfJJpu4/fbbz40dOzawCvEPVapUsXM6DLaNN944KGWG85wbHv/+++/uoYcecrfccouJcxYjd911l7vuuuvc888/H9SO58UXXyyweKEfDRgwwB1//PFZnefAZ916661u4sSJ7tVXX3Xt2rUL3snMJ598YjduZmH06aefmqCZMGGC9WEGf46DZywJ6n3xxRcm6C655BJ38803u8suu8x17drV+iyP6EpZiLIAC2gW6zvuuKPbcsst7dz95Zdf3IIFC1yNGjWCWoVTu3Zte2auY65kjuK8p1+/+eab9l4StDn11FNtazvOgPnz59uCad68ea53797uoosuyjgG0P9wKjCXtm3b1gRWuP8xz5bHRUqxRBIDPIMaq1lEDitC/tEMvNtvv72rVKlSUDMzVatWtWfCCNttt53r3LmziRt+AITHgQceaO+H+eqrr9y5557rbrjhBhM26623nttwww1d48aNbbDlDvqckJxEDOjVqlULWq7KpZdeap8L1atXN2HE5/No2LCh22abbey9JL788ku35557ulq1atn/Yf/99w/eWeFOFQKaNm3qZs2aFZRyBw/l0Ucf7Xr16uX+97//2SLkggsucOecc44NdjwzCCbxww8/uC222MIdeeSRVmZleuKJJ7rffvvNvffee27rrbc2eyYYaDfYYAPzSNFfEVhnnHFG8G4y1L3xxhvd119/bQMqg/R9993nzjrrLFsItW7d2hY29NkoeKVZtDRo0MC8aIhE+liXLl2s31577bXWZw8++GC31VZbBa2EKDsccMAB7uqrr7bXLBKYU5hrJk2a5E466SSzZwNzJX2JOZZ+xxzFud+kSRO31157BbVWBYfGnXfe6T766CPrT/QjxpFmzZpZXyaaQzk6BvA5LIpoy/ckPIczhDHkyiuvXKX/Me+WN4olkoil8s9idYtAwt324Ycf2g/ASQF4lBiEozCojxkzxgZ+VsNeDf/73/82dc17Q4YMscE7LmxXs2ZN98QTT5iKRRghpPbdd9+VoYTddtvNBFYS1GPA54QZOHCg++9//1tA0HDSZKuK8VhRPwrtGdj5/wiBgEYoFAU8lYcccog7/PDDbXHAoLfHHnu4ypUru6efftodddRRbp11krszq8CbbrrJdezY0bxNDIj0TdqzKmSBURiIKj53/fXXt/qsOvk+RxxxRFAjGUIBDLT0Ez6TvsHCBWHH8Sj/+OOP5tWNwndm4URfOu2009wuu+xi353xh741YsQI169fv6wXZUKsSThX6Zt16tSxBTWiA48n8w/vIZYAz04czIV4bAkn40HGG017L7I4Dp5YFiBJ0D9YhJx55pnmyZo9e7YtSBhL+A4IL+Y/nsPgdcJriyMBbzNzLe0YT+h/LPpY5NC/6dflkSKLJGKp5EewOmU1iTo++eSTbWWIG54BlQGRAZgfOAq5GIgnVr4IFJQoK22UKv90RBfvI7wyKWR45JFH7MRhEmDA9qLMe6ji4ATbbLPNbADmhGPy2nnnnYN3nSliVsycTJngf8BgH52gUOCEQxBhmb6HyA8YfAYPHuy++eabwLJidYcnFrFBH2KhQH+J5tBR5lxFCF111VU2MD755JMrc5HoaywQdtppJyvHgZCirwJiA4/vQQcdtDLMXRj0afoJAyUDJqECvi99tjAY3OlLu+66q9t9993tmf6H657+geBikEcwFZZX4WFiQHQxsbCwom2mRZEQpQULfc5rPKAvvfSS9QFy+vC8vP/++7bQJ1cPEeMX+WE4x5mbTjjhBGt3+eWXu80339w99dRTtrggUoItnFsbBucDHiK8Pnhgma+YkwhdM28S7ifchoAqLFzvIXpE/+fz//Of/9i8n81Ca22kyCKJVRsucsQOPxyufwa9oUOHWvIWgznxUzxKuPSiMEji6mdgQ1Cwww0PFPlJp59+unmCvOsO1RrnjfIg2DjJ+EzcjtkMlnxvflw8X0xQTBxhvNs+0+qc1T0TVzQcx99Dgh5/x7HHHhtYRT6CZ4TwL4MRHg/6BqtJBkaSLxE8DFJ169a1pEom/mguH+XDDjvMXuNB5dxlkGTlR9iKQTdTmAmPLH2RNnwWnpswnK98z3feecf6bxwIERYzMGjQIFt9clxCZrjrM8EYgBhkUGVAR2yx6mRg55h8NxZJhKsJH0bhf8QEgseZJFeSVBl/eI1XCXGI9zlTXxWitMDbevbZZ9vCAk8oUQ/Od85n0kUIwyGE6A8+7SMMKR8+b4nx44033rBzvX379uZdRvQwJgD5RVEYO5ib6X89evSwxRlzVocOHcypQJ5gp06dTLTFzdXkErKY4fsS4sZxwDhC/jFeJVJrWMiV10VKsUYVBAb/GH5YBjg8RwxYeIIYtEjOZIWXBD8W+UT8CPz4HIfV8XPPPWexU/7x/LDEPEkQRwzFQZ4FkwQDZ1xOQxIMtiRZH3PMMSb4OME4AZg0UMiQpI5JkkPI8QiDmGOiOe+88+x/IvIbzh8meAZA3O2s6BA6nNMsEhgA8bJw/vIobDWGRwlPDIMUOYF4cMnH4xhxIMhYPe6www5WZjBmZcvCgFA3fQyPJ2IH0YUgi9uSzPEROrj7yYvCzY4rnmRxEqnJHyRPMA4EFf2biYHvwrhAXb4D7fAaM4Ew6PvBPgyrXvIvEFSEtvk/0Xf5LgzcfH8WJUn/AyFKE7zIeH5JtGZuof8jOvDksNGIMDOL9EzhYhwIeKHwPJHigleK/oNgYt5i4UIyON6kaNiNHCIWRvQ/FlQIJo5He0Jp9Ef6PmNU3EKDxRXzOP2Pz2C+xePFZ9H/WPDRFxmHyiXpHy2V/sek1l13XV5mzbhx41LpH9lepwfKVFrcpB599FF7TgukVJ8+fVKdO3e29+Po379/qnnz5qn0gJ1KCyGzTZw4MfX666+n5syZk0oLjVT6hzN7YaTVcapfv36p9A+Yatu2rdnSJ5MdO4m5c+em0it7e83nU06fbKn0StbatmrVKvXiiy+mzj33XKsTJT3oB69WtIf0yZa6//77U7169bJyUeF78ZukJ4fAIsoDAwcOTKUn96C0AvpKmzZtglLhpMWV9b1hw4alRo4cmUqLg1STJk1S6UErqJEM/Y166YHVvkdaDJmtMNILh9Tzzz+fGj16dOrvv/+2c//kk09OjRo1ys73xo0b23eqW7duKr26TE2YMCFouYK33nrLviukB+jUs88+a6851uOPP56aMWOGlQvD9zOgv6cXMqmzzjorsIjyRFqAp9KTblBae2FeGT58uPUZ+jrzU1q0rJxz0uIideedd6bSwiZoUZD04iHVtGnTVM+ePW1OnTlzptmZY+jDaQFWoF/EwVzKnAbvvvtu6rXXXrPX9P/04si+TzaEPye9wLJ25bX/oYeYg6HIniQ8NqhgYpmoStQrORe43jbaaCNbGTZq1CiovSrEP1GyuAs9rHDxzOD6J8TGKhlPD3lJmSBfgnAFbku+Tzawsq1Xr15QWrFCJ7zXrVs3lxZqFgIgF4rwQBxR1Y9XDC9B3PVj8C7Frc6FyBXC0my9ZcXH+Yp3h40HeIDoN4Vd74h6eE7DLvpMkBhNuOuUU06xrb94wO644w7zTOHJIjzGypgwOXkKeHSiHlRWoPRPwCvEipjdavwd9JtsrqcG1CUkyGUDCF8QssO7zIP/C55tIcoSzCtcuoPt8eTM0jcIE5NjxHvk9JFfRzg7DuYUvER4gbynlPM9PXdbjiHeXOpw/vMZeI2isCuNUBrQ/8g7Sgse88rSPtvrJBEd4lIGjAd+wwXfBU8Zn09uZHmkyCKJnCJikiSMElrC/U8yKP84BA5ufQazXGDgJ3xHbhE/HGE4BlSfcJoELnvEWS4k5XDw/fkOhD2YBHy9wk4A6jLxxEFII7prQOQnDG4MUri3swVBz/lHDhN5boCwwE2O25x8J1zeDJK4vRETJQWDqs9FIt8Pdzt9n9A6oTO/nTkXSP4m1Eg+BGED+rsPb0dB9JH0ythCKJ/Bnt05/L08CNuR9E0IgRAgoT0hyhLky7I7EzHDxgzEvXcg4ARgsZJ0/seBU4C+Q1oHYXIW4Zz/pK5ks/BBNNEHcVAwL7GVP2l+Yu5lZxz9j7wo8puY68L9j81Lvv+VR4qVk8QAiTAggZOBzntL8ChxMrCyzbQ7DG+TH9T4oX2idv369c0rxI/JDhxWvpnYZ599glcr1C4Db7b4fAkPPzjXoIkeozgTD8fixBb5DcKIXZSc5+GdlIVB3J+8HQZXEpURCgxabN9lRym7zfDwsELl+Axi2UIeAX2PXXfPPvtsoWKePstgyPcAVpZckDUXyOdjFysLK8QWg3VSPhG77/iOCMGWLVvaa7xh/L08GOxJjGfSIa+DhZYQZQnmSTwu5NySJM18gPhnAwMRGDyqffv2DWqvCvNTWIDgySHKQa4QOUXk03L+M29ms1uV3eaIHxZA9B02ViTlExExYTxBjOH1oszYFe5/9GH/+eWRYo8oDKqs8Ni5wz8QdUtiKe5DklL5EfiBk5JB+ecjQHA58g/3ieB4kVg540kqLIQQhhMKZfz444+bK7Ew8Fh5TxVhCxJE8SLxt4wbN87+PrZox13GwEMCngclT/I5q1++Pw8uURDe+i3yEwQFl7zwoSfIxvPBwMpAhDgipIQ4yhW8u/Qznkm4xN3Og0ROrjdGP8CLmmk1SD8g4ZSdMNQj3M6GCrzJuUACO21Y0NC/SVxNgiRRPzh76OP0Sx6ELBCR9DNW0ixy8NYJUZag33G++qtUs+jBmcBmBRYcXAvJX6A1CsKHjUwsLjj3mSOZZ+hDbGDg3Kcfcsxszn3mWXZ20/+Y8+J2xHnwGrMjL+yhCvc/Umv8PEf6DAnq5Y0K6X9qin8AajfuB0qCfxQ/DGLIryRRuKhkvD8ers/CShVvT9xF59h9Q8yVH41QAnURVv72JuQoFbbjJwoDOF4ocpwKg+/Ld0SVk4fkvVmAK5SQBjAY4zaNA88Tg3MYTl5W3R6EIydUNpDbwcTId4m7R44oP7AYwJ2NpyQT5P2xAOG8DoeWWRAw8BYGYoztu0k5doVB6JywMWEyD9+bxQ2LgGxh5yuTA/d6pD8wAdD3s4H+xKBOnpQP8TF2+XwLD8fLZkUtyi54BPkNox79tRH6DrmDfgcbooTcOvqSFx/Mg/5SGv7SOlHYfYag8vl4XOOPLfvMj3hlvXc3EywkEGRcboRj8Mj2tij8DUSNmJN8/+M38heO9uDlQk+szeBZw7GDuC2ySIqDE5rVaK7wRRjE/WqR0FQ4hFYU+APX1A+FKCNPoqSQSMofWPnxG+MxSQJxwgDF9VWikz8hvGxDd0XtEyxc6J8sFMJg45i5CBIEDp7hwi4QK/Kb8iSSohR1noRwWyIWRHFy6X8sNPAoRy9dIwqy2kSSKBkkkoQQ+Ux5Fkmi7BMWScpyFEIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBhW3rsNsr0bt1i9LF682G5oyv2LatasGViFEKL8w02fP//8c1ehQgXNSaJU4PyDAje4BU5OTkwPZSgJmy9DNm3DZYjaktpBrsf3dcDb42yQ1LYkbdm2A2/P5fiQjS1chqituMcHb/NlyKZtuAxRW1I7yPX4vg54e5wNktqWpC2uDnh7nA2S2hbVFi5DLscHb89UL1oHsvmMcBmitqR2kOvxfR3w9jgbJLUtSVtcHfD2OBsktS2qLVyGXI4P3p6pXlFt4TJEbcU9PnibL0M2bXM9PuRii6sD3h5ng6S2RbWFy5DL8cHbM9Urqs2XC4gkvBYPPfSQvSFKF1Rsz549XeXKld29994bWIUQovzDJHXBBRe4ihUrugcffDCwCrHmaNOmjVu+fHlBkcQJ+ccffwRVRGkycuRId9xxx7lNN93UzZkzJ7AKIUR+wKJ93XXXdb///ntgEWLNsd5667m//vrLRJISt4UQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECKGNSKSnn76abdgwYKglD3cOyVMtMyNELO539yMGTPc6NGjg9IKrr/++uBV8Yl+LyFWF9OmTVt55+qiwv2Ibr/99qCUO999913wqiAffPBB8Kpw7rvvPjdkyJCgJER+8NZbbxXov88884z78ccfg1JmuOn5Tz/9ZK/nz59vz55s58JPPvmkwOd36dLFzZ49Oyhlz6OPPlqkOX1tZI2IpEmTJrnx48cHpez59NNP3ZQpU9ysWbPczTff7N5991332muvuYEDB7revXu7s88+21166aVB7WRGjBhR4PO//vpr9+2339prfuivvvrKXhcGk8DUqVPdhAkT7PHQQw/ZZHPqqacGNYRYvTCoRvsSN0G+9tprg1LhDBgwwG288cb2GoFf2PnP8cOLjG7duln/69q1q7vhhhvs+c4773Tt2rVLXDBMnjzZLVy4MCg5d+GFF64UWwza48aNcx9//LGVhShv/Prrr/b87LPPWj9gDmOxwg18R40aZe8VxvTp0x03owcEE3PQsGHDrC8idjp16mTvZYK5cPjw4UHJuZ9//tlu5pore+21l+vTp09QWiHamFfLI2ss3LbzzjsHr1awbNkyEz933XWXe+GFFwLrqlxxxRXu9ddfd59//rndGfqbb75xNWrUcP/73//cPvvs41q2bBnUjOfPP/+0E5LP7969u7vnnnvcoEGD7ERF5PBDM4BHV+czZ860ExDFfOutt7pbbrnFBNKTTz7pfvvtN9e3b18TRx07dnR16tQJWgmx+li6dKkNjJz3wDn74osvujPPPNNtu+22BURIEr/88oud9/Q/BtdmzZq5xx9/PHg3ns0339wtWbLEXXPNNVbecccdrf9dcskl7vvvvzeB1rlzZ3fMMce4ihUrWp0oCCL68nPPPWciq1+/ftafzznnHBNaTBR4fIUoj7zyyismUJi7NtpoIxMUCB76T+PGjYNamdluu+1c9erV7TXzIfPQIYccYn2xSZMm7r///a+9l4ktt9zSVapUKSit8CpXrlw5KMVDv2QuZN7s0aOHu/vuu21hxXiEWKPMfDhy5MigRflijYgkBs51113XXiNYWHGy8kT8vPzyy2733Xe396JssMEG7qijjnJ77rmnDcIVKlQw+yabbGJeJVQ4rzOBuLnqqqvc22+/7dq2bev+/e9/u1122cXdcccd5p5koOcE88f2VK1a1e2www5up512crVq1XJXX32122KLLVzz5s1d7dq17TiccHi6qlWrFrQSYvWBoEdgMLjSbxD8DFqsDDm3GXwzQX956aWXTFS1atXKNWjQwATPTTfdFNRIpn79+vYA+hyrzw033NDEGV6m5cuXu6233treT2KbbbZx++67r1t//fVtYXHxxRe74447zh100EHuyiuvdCeeeGJQU4jyBYsRQuVAyOv44483gYJHt0qVKmYvDOYrFhP33nuv9T3mIeYe+iJChnkyV/D8Mj/jsMAxEAeLpM0228wdeOCB1kdPOeUU8yQdffTR9tn8XUR0GIPKI6tNJHEC8I9HCBFHRWkyGKN2H3jgARsUERy77bab+89//hO0Ksgee+xhEwLhOgZXGDt2rLkIGXA5uaiTBEKKQfiII46w9ihvJpoWLVrYQI/QGTp0qE0ahBTCcOIgjphEwiDyOAaii+dXX311FS+ZECUN4ShCzawC8cZwPiMyEPJRgR8HAok+iDhioCMnjwUEXqAk708YvFS77rprUHLmVSXMxooWj1Y2+Qn0JSYFciDIRyKEzgqbFSoeWRYzQpQ3CLHhvSUKggeVfjxv3jxbYLMYZ5HBXEJ/pJ+GQfzggcVbQ5ideQsxQjuOSSSGBfyXX365MqQXxUdtOBYCjT5HdIQxgJQWHBCEv994442gRUEYc4488kjr/zfeeKP9PcylLIxOPvlkm4vLM6tVJBFG23TTTU0k4aq/7rrrzE2HcIL33nvPXPSFccIJJ9hACocffrh5l1DSnFxJEwQnBCqXz0KUMcgzMDO5cMIx0RCKwzuEmuZ4STAhsHLnb8KzdP7555vY4xmRlqmtEMWF849BFU/R9ttv704//XTz4OQCA2zr1q2tTzC4IVZYnCQNjJ4PP/zQxAsDNSLLwwIDgcVqEi8x+RKFQc4RoT0WOXhzzz33XPM+4bJndczALUR5g/5K9ITFwWmnnWbzCAuTL774wkLOixYtsrA5/Qwvaxj6x2233WYLosMOO8zmL/obC442bdq48847zxYqeHKS5lLmtyeeeMK9+eabNh9zjPbt25vgYT6l/w0ePLjQ1BXAg1S3bl3LQSJNpTgbQNYWVptI4oe84IIL7DUnAR4b8hpwETJYMlATo83GxY6QQXUDOUJ4fhBNnHA8c9JF4eTy3iYmB2LBZ5xxhjvppJMsrstgjcDBVUhuRvTkxLVJDhOfh7pHRXNCA6oc0fXRRx/Z37XVVluZXYiSBoHEwoLQWBQGKoQ+q1MGPEQ7A2ocDIasCFnFct7SlhUpYqVDhw4mXOI44IADzONz0UUXmUfWQ24Uq19C4qxU8Sb98MMPwbvxsNj5v//7P3PbswmCJHT+NsYCIcoziCA8N+Tm4flhLiQaUa9ePQtnxaV8RJk7d64t9MeMGbMyr4j+T7iaYz388MM2Z0WjIuQcIaKY/4iiMP/50Pzee+9tz9nCvM58jmeJeRgvFMKpPLPac5LwJqGka9asaQMkNGzY0BKgUcOFJY1xEvDwK1WS1Bi4UeaEw9hSmZSwysRBSI8TissQkFjGD0yyG5MGwomTk3hrFFbchAT5PE4Cvj+Ci0kGYcQKG4GF+i/sbxCiqJBzEF5IIC4Ic+EN5XzG7c3Cg90tiBY8PnEggtjVhihiwwGCn3OcVSUrUAY7RBbhZL8gAer5xYGH/kYYnIUDfRGvLK8zeZPod/Q3YHGDF4nVMaKOpFAhyiN4cZgvDj30UFtck4tH7g8LBeYWIhQ4C3y+X5TFixe7Rx55xHJnmW/IB0KgMA4wBpB+gnhiLj322GNtzkJ0ZQN9kogL3xHHxfPPP78yyhMFRwSfhzOB8YH5m/wodpiX90VOhfQKMMWAh/svm+ss5ALxTn4AlC/bE8lb8FuPce2hrJOStj38MJwErFRJdps4caJr2rSpDa4kqrE7JglWqX5g5jhMBsSFUfOsprlWS5xACkN9joOwIpuf78/fROjw/vvvD2qVLEx+5FLhGo2uCkT+wioQoRMmzhbGb9/H24m44XxmFUteAosVBjjyGTiXGXjx2tInGRPCEK4j5IcHivA5bv6nnnrKduaQG0F/yPRdcMuTd8Q1YcijYCXLipb+TE5fLpcwEOUfhDlei2iOztqGT9ZmIU3fYGFCfyNMzQL9scces77FXBQHgoSwXKNGjSwkzTxK2O6dd94xLxTzImKF9+nXeHbD0N8JizF3sZOOMYAHfY6drmw+oq/zvfAq4c1ChMWBUPLjAp/PZXT4XOZQtAPjA+9zzLUd/h/M8/zNq82TxI/y2Wef2UrT4wUSPxpqN3y9hiT40Tgx/PZjYCBnRey3QifhBRIwGaCa8T6xYuaE4wTKBEobzxGTBycAfxMnLWECksk5BruN2DEkRFkE7yerTzynrF4RRvQBvKx4j3CZ77fffjYokOvnN0tEYSAnrMeKGDguixcGcBYRDCiZoC6fQS4T/ZKET4QWg/zll18e1BKifMHGChYp5AMx/+AcQDAhkAAPLPNZEixY6CvhUBzzEdENQmv0Y+ZZ8vnwONOfwvhoDf2WRQrzFZEdvLgka7OwYRwgnE7/jxNI5DzhVCDpu3///ragInyI8wMvFzv3cHjwHcqjV3i1iSTULf+8KOQvIDLYgs/JweBbGAgqBlgPbn3ck6xgs/W0cPIgzgj/kahGXDiTuOFEQExxcnEdFwZ3wn6sbA4++GALU5DLgXhj4hGiLBLuN/QjEr/JX0A0AatLdpgWBgM8IgsxxACNl5h+yCqS/L5MO+RYsdJ/PCx62BVKv2FgDX9HIcoTeMTYUYr4YKHO5gc8QMBcwg5sFiVJKSNxsHinz+Kx4RIghNjIu8U7Fc1rol8SlWDDEWKpV69eFt4DbITrCJfH5fV6WAiRG8n35TPIZSTHiQRuFjjkBftrEBJCLG+sNpGEiImuSBFIuNVRsLhSyRfCTc9AHQdhsWi8k9wK4qa4C/mxyHPKZvsxJwvqmQffAfGDik6C7H/yM/bff//A4iy8x8nOyUWyXGGJdkKUNAxUrOp4sGMFoUJiNcKfwRhBkgnCXniP8IjiIufSAtkkbxIao0+wiiWvAm8U+Xg840mibydB3gV93cNWZTyzDKgkmxLCE6I8w+KEjQ8IHCIoLNgRF/QB+iMeoSTwwuKJAvoZEQ1gYxLzGvMQ/T4cOYkD7w+LHbw+HrxK9EfGDr5bHHi96L/MfSxo+MxMi6LyxmrNSfIgRhjQSSyNXleBay6QdMo1lHgvnASNp4fBnKQ1wHPElmViuz67n51pxGhJWiNvKEm48BmcqCSeMUDjWeKkpU1hkNTGREA+hY+TE49FNHEc3J0kz5511llBi+KhnCQRhQRtPJv0Fy6dwcqP0BjXTeGcjyZXR8ELyjZfBBEPBltED4mhJGFmA4KMfApWkGHoDyw+OBbu+DD0GVbLDLJ8Jp+FN4q/he/NjjhWwXin6I+ILq4sTD8T+Ut5yUkCcoWYy+ivfkHC4oD5zu+MJgzHJg28tXFb8VmcEC5jgc/lZ7jsDHMml/EgR5Ay7TlmHOQgshhJcgywGYScXS4zkAn6LnMujglEGZswyiOIQZ+TxCSfSp+QqfQJycvVwmuvvZb6/vvvU+kTPrBkR1q0pd5+++2glEqlf+RUWlUHpdxID+LBq9zhc6dNm5ZKK/jAsnoZMWKE/SZVq1YNLEKkUmmxHrzKjVmzZqVGjx6dWrhwYWBZwdSpU1PpwTcoFc6kSZOCV6tC3xgyZEhQ+odx48bZc3ohYX05LfasHCY9iaQeeeSR1MCBA+37rKl+JsouaQGdSk9UQWnthn4bnvvSC/9UWmQEpexJL8wLzIfdu3cPXmVmwIABqQkTJgSlZJhvsyG94Eo1adIktWjRosBS/kAPMQfDGvEklQXSf+ZaEx6TJ0kIkc+UJ09SabM2zX1lhbAnabXlJJU1dJIIIYTINzT3FY+8EUlCCCGEELkgkSSEEEIIEYNEkhBCCCFEDCaSuBYRW4RF2YALbPKbcKkCIYTIN0g2Ls8biUTZBj3kr9FoIsnvJBBlAy5gyW9S2H3lhBCiPEKysa7ELkoL9BBzMCjcJoQQQggRg0SSEEIIIUQMEklCCCGEEDFIJAkhhBBCxLDytiQ8uEGlKH2443ODBg3s5qXcoVkIIfIFdrZxI3JuYp50Z3ohVifcMJhbkvBYKZKAkzN8CXPKUBI2X4Zs2obLELUltYNcj+/rgLfH2SCpbUnasm0H3p7L8SEbW7gMUVtxjw/e5suQTdtwGaK2pHaQ6/F9HfD2OBsktS1JW1wd8PY4GyS1LaotXIZcjg/enqletA5k8xnhMkRtSe0g1+P7OuDtcTZIaluStrg64O1xNkhqW1RbuAy5HB+8PVO9otrCZYjaint88DZfhmza5np8yMUWVwe8Pc4GSW2LaguXIZfjg7dnqldUmy/n1b3bhBBCCCFyoUC4TRfvKhuMGDHC1a9f38Jtc+bMCaxCrBkYB3A3b7DBBm7BggWBVYg1Ayv5ihUrWrjt999/D6xCrDm4Rtcq4TZOSomkssHIkSPdcccd5zbddFOJJLHG4WqzlStXNqEkkSRKA3+BY4kkURogkv766y+F24QQQgghkpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIG3butDJLv926bNm2au/vuu90OO+zgttlmG3fUUUe5+fPnuz333DOoUbJwQ81Bgwa5rl27urFjx9qNhfOZtenebb/99psbOnSo+/HHH93RRx/tjjzyyOCdgsybN8/17t3bVa9e3R1zzDFu++23D97JnaVLl7q33nrLTZgwwcrrr7++W7Zsmdt1113dKaecYjcLLwpLlixx9913n3v33Xftb8pndO82UZqUu3u3LV++3D3//PM2yIi1m1mzZrn69eu7M844w11yySXuhBNOcE2aNHE///xzUKPkYYJDKH355Zf2LNYeELQNGzZ0N998s2vTpo0NanHcc8897tprr3W77757sQQSICCPP/5498gjj9gihvP0rLPOcs8++6w74ogjTNAXBRapv/zyi5syZUpgEUKUNmVSJE2dOtW1b9/evAiPPfbYSm/K9OnTXY8ePcx+44032uoRFi9e7M4++2yb5MTazcCBA82TcfDBB1uZSfCuu+6y16sLVqycU2LtZLPNNnO1atVy33//vXvuuecC6z/gDRs3bpy9xjtWEuAtwoNUrVo1K3P+PPjgg+7DDz+056LA36HzUIiyRZkUSf/+979tpfbrr7+a+Nl8883NzgBy+umnmx23tl8RbrTRRm7hwoVu//33t7JYe+G3ZLK75ZZbAotzhx9+uNt5552D0orQSZ8+fcwDhIC688477XWY9957zwT1dddd50aMGBFYV3iNXnrpJffpp5+6hx56yL3xxhvBO/9AOIU6gwcPtklPlH123HFH17hxY/MoRenVq5d5eqLgNcT7w/lz0003uUmTJgXvrFh4PfXUUzau9O/f397Hy5mJKlWq2POiRYvs2cPnvPnmm3ZOE/KLhpAIGd57773ujjvucLNnzw6sK87D119/3c5DPGR8p2HDhlk5DN8RrxYLx/C5DrR59NFH3TXXXONGjRoVWIUQWZPuwKl11lknlV5N87LMMHToUPte6cEhsKzg22+/NXt6kgss5Y/0QGd/Y9WqVQNL/pCeQFL169e3v79p06ap9KQRvLOCmTNnpho2bJiqXr16Ki2AUhdeeGFqvfXWS7Vs2TKokUqlJ5ZUWlSl0pNH6qOPPkpVrlw5NXbsWHsvPeGRg5dKi3Brc8kll5j97bffts9MCzArX3rppal33nkntXz5civnE3/88Yf9L9KCNbCUfZo3b55KC2Mbx3j2/Pnnn6m0QEpNmzbN/qZPPvkkeCeVSgufVKNGjew3fuCBB1JbbrllKi1Y7O/v0KGD1U8Lj1THjh1T1apVS6UXYfaep2bNmqnu3bsHpVTq+uuvT6UXcjZGhcHO+ZcWPanTTjvNvo8nLahSJ5xwQurzzz9PzZkzx45Zo0aN4N1/xsElS5ZYOS34rOzPywULFqTSC8bUjBkzUlOnTk1VqlQp9fLLL9t79J22bdum5s+fb38/48nw4cPtvbJOhQoVrF8LURowjtDPoFzkJLG6ql27ths9erSVSXyuV6+eO/fcc80rAaywLrroopX5AqzSevbs6W677TZrS1hPlD4kzA0ZMsRWvjynJwwLwaXPVXs/PZG5E0880VbI559/vnkJ+A2ffvrpleHWH374wXJPWNnvu+++timBEC6ceeaZlhSanqzck08+aV6EMOS3cawrr7zSkoDTg3XwjijrHHLIIa5OnTqWgO8hV5Gctjg4T/bbbz/7jQ866CDz4sydO9fCr+TEAcng5DPhcSJRm3MyTFpIm8eScB/n5Pvvv1/A6/nTTz+5tGixY+O1JDzHJgFSBwBvJh7xPfbYw6VFjHnOw9AfwkTDcbQnEX2rrbayxPHLLrvMzmFgTCMkyLg4ceJES1rnbxFCZE+ZF0m4wMOPuLyj7bbbzo0ZM2blRIpA2nrrrc29vO2225qNHIJTTz3VdoxBu3btrHzFFVe4Ll26WA6Uz3ESpQsDO6EDQma77babTVjplX7w7grILfHhVj+hffXVV/Z83nnnuRdffNH17dvXpVf6tkshij8Povhcty222CKwiLUJ+vLw4cPd+PHjrUzYlMTuOAhRdezY0c4RBHMciBcgIZtxJjr+sPOS841NI2wuoE4YcqE4n2vWrGmPiy++2E2ePNnEPrAA8J8BCPhcQID5MQ4IN/q/F3HEe/6zEXiEqYUQ2VPmRdKMGTMsB8k/h2P2nvDKzfO///3PvEg+p4TBpG7duvbax/ZJ8iSnhW3fDHbffvutvS9KD+/5A3LMGOj/+9//ruLxCeMnHA/nCUKZiYFJs1KlSsE7hUPybLdu3ZSLtJZCP6af8xviRTr55JMTt+QjYBo1amTeG7zOmeAYeHriIGeSZG08RNF8IfKPuKQFXh4EPw9yLvFuAuNZ0o68bOD40cUd3jDgPfKo/OfyUGK4ELlR5kXSsccea+5k/5x0HZQoTJJMng8//LAlUjKQ+VUabnYSePEesX33hhtucK+++qqF3UTpwmo3nNiKwGnVqlXGa3gROmXS8Yn7hOoI07H6z5UOHTrYuYPILuvXCBLxtG3b1hZFeFUIqyaBMCJkSxiqMPBSc176XZdROGcuv/xyG1MQ6R4WcITW+vXrF1iceTn9Yo8xiu9aVDg+x/b9g52h/rN22WUXS9rmsgIeeZKEyI0yL5KKwwUXXGDubAYGdst5yEFgR4jPYQIGF3mSSh/yKdhJ5PMqgDyP1q1bB6UV4A30+WUDBgywvBMffmMy44J8TE7PPPOM/bbsiCMcx2THI3x88GU8Bn7S4TOLs8oXa4bo7+l3vjZt2jSw/PP7hutxnnDRRsYCvECAJzMsKryXhnwkhDiLNQ/HCh+PXChEC94rL7DZmckD8YR441pNhAO995NzjHGI3Dra4NUmdIf3CQiXMV6xy43Fnb/I5GeffWbPiPrPP//cvGe33nqr7fDz35EcTP5GPv/666+3sGCSN0wIEU+5Fkm40clHefvttwtcHoAwDINZp06d3Ndff202tvnG5a6INQv5GYTXSDolFMrgzpW32cofhomDSYF8EiYM8o88V199tYXNWrZs6fbZZx/XoEEDu0IyHgNCrM2aNbM2fqs2gohyixYtbJIiWRbRxaT4xBNPZPRiidKF7fMIYfJ+2GaPaEHoIjoQCYDoQZjw+zIWeOHDVd25cCPjwDnnnGNJ/niUw54lPDGchyy08PhwTuC5JFxPsjdeI3/pAN6jHuE0tvuz6MJ7TV2EC0IH8cNWfw/C5fbbb3dcaZvvd+CBB9qCznu9ucr8//3f/7nOnTu7+++/34QflzPw4uywww6zpHKEHv8Hci333ntve49zH68V1xrDQ8vfl8mzJoSIIb0Ks61uZe0SAOmBw77XF198EVhW8Morr5g9PTAEllQqPcGZ7emnnw4s/1CnTp1UepALSv8wcuTI1IYbbmjbw9m2m17RBe+UPvl8CYBs6N27t10CQKwe0qLQzr+16RIAJc24cePsf8DWerHmSS+CdAkAUWqU+UsAsF2bfBJ2rBEmCV9xm1URdtzQfkWIRwDbTjvttMqqn+25fvdTGNzTJG6S4MkF3op6lVwhhBBClE90g9sySL7f4DYT5CER/iB8xnWuCE+IkoUcrrXlBrerA5/8zCUCrrrqKrtUiFizpFfxusGtKDVIufA3uJVIKoNIJInSJN9Fkih9JJJEaRIWSeU6cVsIIYQQoqhIJAkhhBBCxCCRJIQQQggRg0SSEEIIIUQMEklCCCGEEDFIJAkhhBBCxCCRJIQQQpQCXOJg1KhR7oUXXrBbyEycODF4Z83y888/262fuG2PKIhEkhBCCFEKcM/BAw44wG5U/Morr9j99kqDyZMn2/0GdV2+VZFIEkIIIUqJKlWq2IWDt9tuu8Cy5uGmytw0XKyKRJIQQghRxli4cKHdX5QrPxOOGzp0qF0BmiviP//8827YsGFu+fLlQe0VjB492p7xSvGI8tlnn7m+ffu6AQMGuNmzZwfWZL7//nurz216fvnll8CaX0gkCSGEEGWIb775xm5N1blzZ7tXJXlLrVq1srwhyuPHj3ctW7Z0/fv3t/qzZs1yJ510kmvRooW7/fbb3W233eYaNmzorrnmGnsfnnzySXfhhRe6U045xcJrhx56qFu6dGnw7qpweyyO2aBBA7tl2Z577ulmzpwZvJs/SCQJIYQQZYhdd93VHX300SZ+2rdvbzfzPu2009yQIUPcxRdf7Lp16+YaNWrknnnmGau/5ZZbmphZvHixiSsSsBFYd9xxh5s+fbrVITF87733dlWrVjUx9d133618Lw48UTvssIOrXr26O+OMM+zm4pMmTQrezR9W3uAWtthiiwJxyfRb9lwSNl+GbNqGyxC1JbWDXI/v64C3x9kgqW1J2nCvckJi33zzzc0e1w68vbifCVFbuAxRW3GPD97my5BN23AZorakdpDr8X0d8PY4GyS1LUlbXB3w9jgbJLWNs3lXPINvUr1wGXI5Pnh7pnrROpDNZ4TLELUltYNcj+/rgLfH2SCpbUna4uqAt8fZIKltUW3hMuRyfMRBPt3gFvHSrFmzAp6fK664wo0ZM8a9++67Vu7UqZN5gPDwQJs2bdxPP/3kXn75ZSv37t3bxBM71WDatGlul112sVAdgmrJkiX2v0YscVzqT5kyxQQZVKpUyQ0cONA1adLEyniPli1bZp+PJ6t79+7ujTfecPXq1bP3yzPhG9xyUqbWWWcde6T/gStf+3JJ2Xy5sHreFi7H2ZLaZWuLlqP14mzeHi2XtC1aTrKF7ZnqFdUWLsfZktoVxebLhdXztnA5zpbULltbtBytF2fz9mi5pG3RcrRenM3bo+Xi2MLlOFtSu6g9U71oubB63hYux9mS2mVri5aj9eJs3h4tl7QtWo7Wi7N5e7RcHFu4HGdLaucf6YmK6Skv2GuvvVJdu3YNSiu4/PLLU4cffnhQSqU6duyYqlu3blBKpVq3bp1q0KBBUEqlevXqldpmm22CUiq1fPly+z+mRZGV06IpVadOndTYsWNTkyZNsv/31KlT7T2oWLFiavDgwUEplZozZ06qfv361n7BggV2rBEjRgTvlm/SAt3+P7DSk5Q2uKeeeiprpZ+rzZchm7bhMkRtSe0g1+P7OuDtcTZIaluStmzbgbfncnzIxhYuQ9RW3OODt/kyZNM2XIaoLakd5Hp8Xwe8Pc4GSW1L0hZXB7w9zgZJbYtqC5chl+ODt2eqF60D2XxGuAxRW1I7yPX4vg54e5wNktqWpC2uDnh7nA2S2hbVFi5DLsdv3ry5PEnF9CTxv2RuJ1l7jz32cEcccYSFzdq1a2fHqVWrVkZPEnXZcdejRw9LIt9kk03y25OEchJCCCFKm7RwkiepCJ6kqlWrppYsWWLlIUOGpI455hh7DdWrV0+lhVUqPfGn7rvvPpv3J06cmJo7d669H/UkHXrooakTTzwxlRYL5k3iN0kLstSsWbOCGuWXsCdJidtCCCFEKYCnDE/QBhtsYN4df8XtGTNmWFI1W/y//vpre/3rr7/axSenTp1q3iJyhhYtWmSeIg82dsDde++9btCgQbZ139OlSxf3xBNPWNJ2jRo13MYbb2xeIo5JkjberAkTJtgx4bLLLjMvVv369a0uXqW0uFrp7csXVobb0irS/sFCCCFEaUL6Rz6F20oCH27j2kZcS6ly5cqrhDz5f/J/5f/LfM/raJ0w1PH6gGPyTNvyTjjcJk+SEEIIUU5AyOCZihM/eI28yEEIZBJIQB2OB15c5RsSSUIIIcRazLx58yzshvfjo48+CqyiJFC4TQghRJlC4TZRmijcJoQQQghRCBJJQgghhBAxSCQJIYQQQsQgkSSEEEIIEYNEkhBCCCFEDBJJQgghhBAxSCQJIYQQQsQgkSREEci3+xcJIUQ+IpEk8hbuRcSNIA888EBXu3ZtN3LkSLt4GPz444+udevWrl69em7YsGFm8yCQuEkkN5+MsmTJEjd06FB32GGHuRNPPNHdddddrmPHju7ss892kyZNCmo5u0Flnz593CGHHOIGDx4cWP9h9OjR7rjjjrMbUs6dOzewCiGEWJPYFbe5uqm/gZ0Q+cZvv/3mdt11V9ehQwd3/fXXmw0Rc/fdd7vbb7/dymHOP/98N2XKFLsr9tNPPx1YC7L77rtbvcsvv9zKCCr6GOXu3bubDW688caVIuz11193VatWtdfQvHlzN3DgwKAkRH7AIsRfcVt3gRClAeceV9xm3JYnSeQ9m2yyibvvvvvcLbfc4r766iuzIZA6depkr8Mgjrbddlt33nnnmcdo1qxZwTuZmTFjhj1Xr17dnsPgbWKB0r59e4XxhBCiDCGRJEQavDaHHnqoa9OmjXvnnXfc9ttvb56iKH379jWBdMopp9gdtQcMGBC8kwxhttNPP901bdrUXXTRRYH1H7bcckvXr18/8yTFea6EEEKUDhJJQqSpUKGCe+ihh9wHH3zgunTpEitmFi9e7ObMmeN22GEHt/HGG5vwQdz4PKYoY8aMcVdccYWrVauWa9eunYXOKlWqFLxbkP3339+Ode2117q33norsAohhChNJJKECKhZs6arU6eOq1GjhuUPRendu7eJnJ49e9oDT9Inn3ziPv7446BGQY488kh32223uVNPPdV169bNLVu2LHgnnoYNG7oePXq4xo0bu+nTpwdWIYQQpYVEkhBZgMCZOHGiCaXOnTvbA6FEiI5dapno1auXmz9/vu1yKwySx8844wwTTNpIIYQQpYtEkhBZMGTIEMtDikJ+0lNPPeUWLFgQWFalWrVqVuexxx5b5XICUdjVQ15S5cqVLTFcCCFE6SGRJEQA243Z+v/dd98VuAbSzJkzXdeuXWMTuQnRLV261F155ZUr28yePdsuK/DNN99YGQjj4SUiQXzQoEEr85ioE64HVapUcf3793dbb711YBFCCFEa6DpJQqRh6/17771nQofX6623njvooIPsvfHjx9tFIukn++yzj4kYwHs0efLkldv2N998c0vq/vTTT1cKJi4vsNdee9nr33//3fKXqL/jjju6efPmuYULF1rSOGUuLRCGyxGQHyVEPkH/0HWSRGkSvk6SRJIQQogyg0SSKG10MUkhhBBCiEKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKQSBJCCCGEiEEiSQghhBAiBokkIYQQQogYJJKEEEIIIWKokErDHZcrVKjgdtttt8AshBBCrHnSU5KbMmWKY17SnCRKg6+//trOw+XLl/8jkgAjYslDGUrC5suQTdtwGaK2pHaQ6/F9HfD2OBsktS1JW7btwNtzOT5kYwuXIWor7vHB23wZsmkbLkPUltQOcj2+rwPeHmeDpLYlaYurA94eZ4OktkW1hcuQy/HB2zPVi9aBbD4jXIaoLakd5Hp8Xwe8Pc4GSW1L0hZXB7w9zgZJbYtqC5chl+ODt2eqV1RbuAxRW3GPD97my5BN21yPD7nY4uqAt8fZIKltUW3hMuRyfPD2TPWKavPlAiKpYsWKbvLkyfaGEEIIURowSdWsWdOtu+66btKkSYFViDXHXnvt5f76669VRdKff/4ZVBFCCCHWPIgk5iRE0h9//BFYhVhzcO55kaTEbSGEEEKIGCSShBBCCCFikEgSQgghhIhBIkkIIYQQIgaJJCGEEEKIGCSShBBCCCFikEgSQgghhIhBIkkIIYQQIgaJJCGEEEKIGCSShBBCCCFikEgSQgghhIhBIkkIIYQooyxbtix4VTiffPKJW7JkSVBybsKECe6dd94JSpnp1auX++mnn+z1b7/9Zs9huJdZPqIb3Iq856uvvgpepVcN6b6w22672evvvvuuQJ/Ydddd3fTp093SpUsDyz/svPPOdlPEMIsWLXI///xzUCrIBhts4Lbffvug5Nzs2bPdiBEjzF6nTh33+++/uy233DJ4V4j8IV9vcPvjjz+awJk3b5778ssvTZRMmTLFVapUyXXo0MFtvfXWQc1VmTVrlo0XDz/88Mrxq1atWm7BggWua9eurm/fvmbLxFVXXeVat25tY1m3bt3c6aefbuPXDz/84KZNm2YCqk+fPkHt8o1ucCtEiE022cSddtpp7pxzznFVq1YNrM5tvvnm7rLLLnN169Z166+/vvvXv/7lNttsM3fLLbe4ww47zH300Ufu22+/daNHj3YNGzZ0LVq0cFOnTg1aO7feeuvZILXHHnu4J554wuryePnll92FF14Y1Foh0urXr+922GEHO+69997rmjdvHrwrhMgH8N68//77bqONNnKNGzd2559/vuvRo4e7+eabMwok+OWXX1zPnj1dtWrV3C677GLHYfxBbJ1xxhlBrcxstdVWJpBg0qRJNlZR5ruceuqpNjblIxJJIu9hAGJgqVmzpgkjz8Ybb+z23ntvt80227gdd9zRVahQwQawffbZx8RSy5Yt3QknnOAuuOACN2zYMFe9enUTVN988421ZwV40EEHWbv99tvP6vK45JJLCgxcCKbjjz/eHX744bYa7NKli9tpp52Cd4UQ+cBee+3lWrVq5fbcc0+36aabBtbsYEzy4TC80HiT8MjhBWJcyQbaf/zxx+6RRx4xTwpjFSKJ7zJjxgwbw/IRiSQhSgBWbXiY8DZdeumlgTWZZs2aBa9WhNrwRi1evDiwFHxfCFH+IbRDuI0Q15tvvumGDBli3qHLL7/cHgiomTNnBrX/Ydy4cVaPEOVtt91moTUEz9dff23he443fPhwd8MNN1jYLAxhOo5Ne7xPfD5edTxSvXv3Nq/29ddf7z799FPzVuUjEklClBCE5Jo2bWoD3MKFCwPrqvA+YsqDqBo7dqx5ob744guzHXfccfYshCjfPPfcc7bAuv/++13//v1dx44d3RZbbOHefvttyxEi5MaDkD3iJQphsM6dO5vIOvbYY83bTd4kHvDPP//c0gLuuece1759+1U81HiuOTbtjzjiCHfUUUeZl5x8STzceMk//PBDSzs48cQTg1b5hUSSEAEMKKyowg8GiFwguZuE7V9//TWwrIA8JI5HqO3JJ58MrCs4++yz3eOPP275AwcffLC77rrrbMATQpR/mjRp4q6++moTR23atLHwfI0aNUwo+RBaYYwfP94deOCB1pbcSkL9zzzzjGvUqJF5hkgnCKcSxIEXCe8Vu+H8Io5xiHAdu+YGDhzobr31VvN85xMSSUIE/Oc//7EVVfhBTlFJwCqM491000226yTKmWeeacmSJICzs4QdbuGtvEKI8g+7acmRJHyPR4hNJYWBgGFhxpjB5QLwAtWuXdsdcMABFkJ77733XL169UxARWH34LPPPmshN45zyimnmDeJsYhF3ciRI933339voTw2oFxxxRUm3vIJiSQhShBCaSQ7snKLo0qVKuY5CsMOOGB329NPP21u9TFjxrjHHnvM7EKI/ABRkxRqJxE7yvz5821MwQNNqH7u3Lnmle7evbt5gEi2vuuuuxKPSY4SizEWZoTbgPGIsB2hNp6POeYYW9iRWI5YyjckkoQoIbjOyeuvv27XNMk0mER3rgwaNCh45Wy1h1dp3333tS24Qoj84a233rJdZXFw3TZSAsIwluABIln7zjvvtNwhrrN07bXX2vt4kxiL2KkbB2E9Fm3ha7wRutt9990taRuBxThE3hI5k/mYBiCRJEQJwE4StvUfeeSRBa6BlA0kbrLLxMOKkWumcH0SIUR+QE4iXiHyiZIIX4fNw2YRcplox+VHEDJc8wgeffRRC71x5e1sYSzikgLkJXFc0gTatm1r3ip5koTIQ9hFwtZaXM9sp/Ww7ZXYPHkCr732mj1PnjzZdotUrlzZEqyJ2xPPv+OOO+x56NChlk8ArOjYrULobdSoUbEDHLDa40q2JHQ/9dRTJrIuvvhiyw0QQpR/2OjRr18/165du8Di3IYbbrhKXiLjURzcMYPxiDFkzpw5Ng598MEHlmfJMdndluQFYpwivO8hZIdHe7vttrP8I3btkqxNTlI+otuSCFHKsIOF/sdF4Lh0AK5xL7SEyDfwpDIn5cNtSfhbufwHXiQuTosg8XAhSDxBPnmbhdzff/9toa8wHIMdaSRvk0OER4nrIhGK8xeSJOcIbxJ5R4imcHgNWBx26tTJrrNEHhNtWfiRX8n3YDzi0gDcVSAfCN+WRCJJCCFEmSGfRJK/oSxem6KCd5vt/VzxH7g6Nh6opM0jSeCl4t6UXP0fbrzxRruQZD4ikSSEEKJMkk8iqSzjPdz5SFgkKSdJCCGEEAXIV4EURSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgkkoQQQgghYpBIEkIIIYSIQSJJCCGEECIGiSQhhBBCiBgK3Ltt1KhRgVkIIYRY83DvNu5Wz93sNSeJ0qB27dqr3uAWODkrVKhgr4EylITNlyGbtuEyRG1J7SDX4/s64O1xNkhqW5K2bNuBt+dyfMjGFi5D1Fbc44O3+TJk0zZchqgtqR3kenxfB7w9zgZJbUvSFlcHvD3OBklti2oLlyGX44O3Z6oXrQPZfEa4DFFbUjvI9fi+Dnh7nA2S2pakLa4OeHucDZLaFtUWLkMuxwdvz1SvqLZwGaK24h4fvM2XIZu2uR4fcrHF1QFvj7NBUtui2sJlyOX44O2Z6hXV5ssrb3DbvHlzMwghhBBC5DteF5knyV4JIYQQQogA5/4f/xDUMtXXUCsAAAAASUVORK5CYII=[/img][br] 附图 15 M 数据工场的总体架构[br][br] 从图中可以看出,数据工场平台的底层基础平台建立在以 HDFS 和元数据为支撑的Hadoop 体系之上,由于 M 公司及其生态链企业业务场景丰富,因此在技术选型方面,包括Hive、Spark、MIapReduce、YARN 以及 IMIpala 等在内的大数据技术都在不同的场景下使用。中间层为 M 公司自行研发的数据工厂,主要提供数据可视化、计算任务管理、数据管理、权限管理、任务调度和数据共享等服务。架构的最上层为 M 公司及其生态链企业的关联业务,业务的规模处于动态扩展之中。
[b] (三)数据处理方式[br][/b][br] 除提供底层的能力外,M 数据工场也为 M 公司及其生态链企业提供全方位的数据保障能力,这方面的能力主要用于 M 信用卡的风险控制和额度评估、广告精准投放、限时抢购时用数据打击“黄牛”超量订购等,既帮助各业务部门对数据进行系统分析,同时也有利于将数据应用到核心业务场景中去。[br][br] 1. M 数据存储格式[br][br] M 数据存储统一采用 Parquet 格式。这种格式的优点在于其使用的是列式存储,能支持MIapReduce、Hive、IMIpala 和 Spark 等大数据处理技术,并且读取速度快、占用空间少、处理效率高。[br][br] 2. 客户端数据接入[br][br] 客户端数据接入是指通过手机的 WAP、App 等数据采集端,数据的存在方式有 SDK 和服务端 Log 两种模式。这两种模式各有优点和缺点:SDK 模式的优势是采集便捷、数据稳定完整、格式化较为容易并能消除爬虫的影响,劣势是需要前端介入;而服务端 Log 模式的优势是无须前端介入、部署容易,劣势是接口变化较多、采集数据不够完整、格式化复杂以及受爬虫的干扰等。尽管这两种模式各有优势和劣势,但在不同的场合需要两种模式灵活使用。[br][br] 3. 服务器端数据源[br][br] 除前端数据源外,整个处理数据时还会有大量的服务器端数据源需要处理。来自业务数据库类的数据,用 ETL 工具进行批量导入;对服务器端日志等数据,用 Scribe 工具将数据写入 HDFS 之中。[br][br] 4. 元数据管理[br][br] 当企业的业务快速增加之后,每种数据的处理方式都有可能不一样。如视频播放日志、分析师希望用 Hive 进行处理,然后用 Impala 直接写入 SQL 去进行计算,但数据挖掘工程师就要因此用 MapReduce、Spark 等方式去读取和解析。元数据管理就是要做到数据处理方式的统一性,既能够满足 Hive、Spark、Impala,还能满足 MapReduce,同时能有效地节省各类用户对数据理解和执行的时间。M 数据工场每份数据的描述都需要在数据工场上提交,之后数据工场会在 MetaStore 中建表的同时附上元数据的行为,供 Hive、Spark 和 Impala 使用。数据管理还会生成 Java Class 给 MapReduce 使用。[color=#0000ff]附图 16[/color] 为 M 元数据管理体系。[br][br][br][br][br][br][br][br] 附图 16 M 元数据管理体系
5. 计算管理[br][br] 相较于数据管理,计算管理的难度要高得多,尤其是每天面对海量的计算量时,情况变得尤为复杂。为此,M 公司做了以下各方面的优化。[br][br] (1)定时执行:按照指定的时间执行确定的任务。[br][br] (2)手工执行:将不定期使用的作业注册到平台上,在需要的时候,手动触发执行任务。[br][br] (3)任务参数:将任务参数化,提升任务的可执行度。[br][br] (4)数据依赖:依靠数据进行任务的安排和执行。[br][br] (5)指定队列:指定任务运行时所使用的队列。[br][br] (6)指定账户:为任务的执行选择 Kerberos 账户运行。[br][br] (7)结果通知:任务执行的结果,通过电子邮件发送给指定的人。[br][br] (8)数据对接:通过 HTTP 网络 POST 请求回调,直接导入数据库中。[br][br] 6. Docker [br][br] 为了管理好纷繁复杂的计算框架和模型,在计算执行方面,M 公司使用 Docker 来解决对环境的不同需求和异构问题,并且与 Hive、Impala、Spark 等不同的计算模型进行对接,以适配不同应用场景计算不同数据的模型。在不同的业务场景下,同一个计算逻辑也可以选用不同的计算模型,Docker 的使用有效地避免了资源的浪费。另外,Docker 在用户数据隐私保护方面也很有优势。M 公司采用了 Docker 与自身安全策略有机融合的方案,使用户的隐私保护和数据安全达到了业界领先的水平。

Information: 三、大数据平台