Skip to content

Commit

Permalink
created original copy of the model by loading from disk (#2406)
Browse files Browse the repository at this point in the history
* created original copy of the model by loading from disk
* Update fx_graph_mode_ptq_dynamic.py

---------

Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
  • Loading branch information
Samsonboadi and Svetlana Karslioglu committed Jun 9, 2023
1 parent 8966d94 commit 64dfa45
Showing 1 changed file with 21 additions and 3 deletions.
24 changes: 21 additions & 3 deletions prototype_source/fx_graph_mode_ptq_dynamic.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,9 +239,27 @@ def evaluate(model_, data_source):
.set_object_type(nn.LSTM, default_dynamic_qconfig)
.set_object_type(nn.Linear, default_dynamic_qconfig)
)
# Deepcopying the original model because quantization api changes the model inplace and we want
# Load model to create the original model because quantization api changes the model inplace and we want
# to keep the original model for future comparison
model_to_quantize = copy.deepcopy(model)


model_to_quantize = LSTMModel(
ntoken = ntokens,
ninp = 512,
nhid = 256,
nlayers = 5,
)

model_to_quantize.load_state_dict(
torch.load(
model_data_filepath + 'word_language_model_quantize.pth',
map_location=torch.device('cpu')
)
)

model_to_quantize.eval()


prepared_model = prepare_fx(model_to_quantize, qconfig_mapping, example_inputs)
print("prepared model:", prepared_model)
quantized_model = convert_fx(prepared_model)
Expand Down Expand Up @@ -289,4 +307,4 @@ def time_model_evaluation(model, test_data):
# 3. Conclusion
# -------------
# This tutorial introduces the api for post training dynamic quantization in FX Graph Mode,
# which dynamically quantizes the same modules as Eager Mode Quantization.
# which dynamically quantizes the same modules as Eager Mode Quantization.

0 comments on commit 64dfa45

Please sign in to comment.