Loading...

Design and Analysis for Private Machine Learning Algorithms

Ehteram, Hamid Reza | 2020

1765 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 53445 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Maddah Ali, Mohammad Ali; Mirmohseni, Mahtab
  7. Abstract:
  8. The emerging applications of machine learning algorithms on mobile devices motivate us to offload the computation tasks of training a model or deploying a trained one to the cloud or at the edge of the network. One of the major challenges in this setup is to guarantee the privacy of the client data. Various methods have been proposed to protect privacy in the literature. Those include (i) adding noise to the client data, which reduces the accuracy of the result, (ii) using secure multiparty computation (MPC), which requires significant communication among the computing nodes or with the client, (iii) relying on homomorphic encryption (HE) methods, which significantly increases computation load at the servers. In this research, we propose Corella and TaylorMixup as two alternative approaches to protect the privacy of data. The proposed Corella scheme relies on a cluster of servers, where at most T ∈ N of them may collude, each running a learning model (e.g., a deep neural network). Each server is fed with the client data, added with strong noise, independent from user data. The variance of the noise is set to be large enough to make the information leakage to any subset of up to T servers information-theoretically negligible. On the other hand, the added noises for different servers are correlated. This correlation among the queries allows the parameters of the models running on different servers to be trained such that the client can mitigate the contribution of the noises by combining the outputs of the servers, and recover the final result with high accuracy and with a minor computational effort. In the proposed TaylorMixup algorithm, the client relies on only one server to offload an ML task. In this algorithm, we propose Taylor Functionalization Mechanism and Mixup Privatization Mechanism. In Taylor mechanism, the server is fed with the client data, added with strong noise, independent from user data. In response, the server computes a neural network and a number of its higher-order derivatives (e.g., the gradient) at the received query point. To recover the final result, the client combines the outputs of the server with the noise, the version he used to generate the query, with a minor computational effort. For noise distribution efficiency, i.e., negligible information leakage to the server while high accuracy at the client, in Mixup mechanism, the client combines a number of samples which distributed independently from the data distribution to generate the noise. In this research, we propose approaches that simultaneously have high accuracy and privacy, and avoid huge computational and communication costs. We evaluate the proposed algorithms for different ML tasks. Simulation results for various datasets demonstrate the accuracy of the proposed approaches in action
  9. Keywords:
  10. Machine Learning ; Privacy Preserving ; Multi-Party Computation ; Secure Computation ; Private Edge Computing ; Secure Multiparty Computation (SMC)

 Digital Object List

 Bookmark

...see more