Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Come up with solution for large model files in ydk-py #7

Closed
ghost opened this issue Apr 13, 2016 · 2 comments
Closed

Come up with solution for large model files in ydk-py #7

ghost opened this issue Apr 13, 2016 · 2 comments
Labels

Comments

@ghost
Copy link

ghost commented Apr 13, 2016

Currently, git push to the github repository gives the below warning:

remote: warning: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: warning: See http://git.io/iEPt8g for more information.
remote: warning: File ydk/models/infra/Cisco_IOS_XR_infra_policymgr_oper.py is 78.48 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File ydk/models/ipv4/Cisco_IOS_XR_ipv4_bgp_oper.py is 74.33 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB

Need to come up with a solution to fix these warnings

@111pontes
Copy link
Collaborator

We may be able to streamline docstrings. Also, following PEP8 may help.

@ghost ghost modified the milestone: 0.4.1 Apr 20, 2016
@ghost ghost added the bug label Apr 22, 2016
@ghost ghost modified the milestone: 0.4.1 Jun 7, 2016
ghost pushed a commit that referenced this issue Nov 3, 2016
* First commit core api

* Changes in this commit
-------------------------

1. libydk will be library component that will contain the all of the
model agnostic code. The public headers that will be exposed is
        ydk/ydk.hpp :- Declarations for Entity, Service, types,
        ServiceProvider etc
        ydk/netconf.hpp :- Declaration of the NetconfServiceProvider and
        other netconf releated public api
        ydk/core.hpp :- The path api

2. Private header are suffixed with _private.
3. All src is now under ydk/src
4. Samples under ydk/samples
5. Tests under ydk/tests

* Fix ydkgen code for c++ to work with models under test (#4)

* Add test case to use encode function to write to netconf (#3)

* Added get_ydk_path method for Cpp APIs, moved Entity class to a separate header (#1)

* Print virtual destructor

* Update generated Cpp code to be able to commit bgp_sample.cpp

* Fixed path for identity (#5)

* Fix xml serializer encoding of leafref and rpc input data node creation

* Fix for bad access problem, test xr601 model loading

* 1. Move updated fixes to libyang under manradhaCisco/libyang
2. Get bgptest cases to test encode->decode for xml and json (Codec test
                                                              cases)

* Integrate bindings with core (#6)

* Introduce entity, value classes
 * Add hand-written entity classes for a few nodes of bgp model
 * Add unit tests for all new and existing non-core ydk classes
 * Added crud test which configures bgp as, afi-safi, neighbor to device

* Use models path from config file (#7)

* Add CRUD Read implementation plus test case (#9)

 * Fixed const, removed get_ydk_path as it is no longer used (#8)

 * Temporarily committing hand-written APIs to get tests to work (#10)

 * Clean up

 * Added test_entity

 * added << and string override for Value

 * Add == override for Value

 * Added const == override to Value

 * Added == operator to EntityPath

* Coding style (#13)

* Update CODINGSTYLE

* Update and rename CODINGSTYLE to CODINGSTYLE.md

* Added top entity lookup files (#14)

* Added top entity lookup files, rebased on origin/ydk_core

* Moved generated code to sdk/cpp/ydk/tests/models_gen

* Clean up entity_lookup_printer

* Updated to run tests with hand-written files without generating

* Fixed const, removed get_ydk_path as it is no longer used (#8)

* Netconf service provider implementation and test cases

* cmake sample

* update commit

* fix bug with wrong statement being returned for list schemas

* sanity tests

* Ydk core cmake (#15)

* Code generation changes, CRUD service update and add more coverage

* Add bgp sample and cmake

 * Updated to use C++14

* Generate CMakeList.txt

* Replaced GLOB

Because "We do not recommend using GLOB to collect a list of source
files from your source tree. If no CMakeLists.txt file changes when
a source is added or removed then the generated build system cannot
know when to ask CMake to regenerate."

* Update generate.py to create shared library using cmake

* Add .libs directory

* Clean up sample; Fix missig imports

* Clean up samples, provider

 * Fixing memory allocation in provider
 * Handle empty data in read operation
 * Fix ietf bundle gen

* Clean up provider. Still needs more cleaning

* Introduce netconf exceptions. Moved out YDKException to ydk namespace

 * Netconf client throws exceptions in case of error.
 * Added config support to crud read operation

* Gen changes

 * Delete unused files

* Fix cmake file for bundle

 * Move types to types.cpp
 * Move errors to errors.cpp
 * Fix identity
 * Temporarily remove bgp-oper, mpls-te-oper, ospf-oper, infra-policymgr-oper

* Add support for enum type

 * Generate enum class for enum types
 * Add support for setting leafs of enum type using generated enum type
 * Clean up entity path printer
 * Add more timing messages to generate.py

* validation service

* Clean up c++ ydkgen printers

 * Fix cpp bundle/core compilation
 * Add missing pcre link to cmake file

* code review change

* Fix large model files issue by using compiler optimization -Os

 * Add back the temporarily removed large models to cisco-ios-xr bundle profile
 * Generate/compile time down to ~36 mins on mac

* Fix duplicate symbols issue with enum

 * Print enum to string function body in .cpp file instead of .h file

* Fixed samples header paths

* move libnetconf and libyang object files into libydk archive

* Remove unnecessary check for options.core

* cloning from libyang ydk_core branch, use os func to navigate file system

* added readme

* migrate libydk archive installation code to sdk dir

* added missing dependencies in cpp/samples/CMakeLists.txt

* removing leftover functions in ydkgen/__init__.py

* renamed ydk object lib from "temp" to "ydk_core"

* Core doc (#17)

* Migrated some static docs from core.hpp

* Update static doc

* tweak

* Added "Edit on GitHub" link, used forked branch temporarily.

* Added doc for crud service and netconf provider

* Code review update, disable "edit on GitHub" link for generated docs

* Fixed core generation/test path/redirect debug msgs

* Fixed link for Enum/Identity, cpp primitive type. (#18)

* Fix crud test (Still failing)

* Add support for bits/decimal64 datatype (#19)

* Rename generated header files to .hpp
 * Escape C++/python keywords differently
 * Cleaned up coding style doc. Added contributions doc
 * Updated samples
 * Explicitly name the libyang/libnetconf .o files

* Added support for Bits documentation (#20)

* remaining validation , plus repository model discovery

* make boost log link dynamically

* Fix core/generated code to fix some issues (#21)

* rename ydktest gen-api dir to "ydktest-bundle"
 * migrated more sanity tests
 * fix test cases
 * Fix the case for creating datanode for container.
 * fix problem in creating the same list node when it already exists
 * reenable support for bits
 * clean up logging

* Delete hand-written api files and move valdiation test to outside core tests

* Fix ydk cmake

 * remove-private-as leaf not working somehow

* enable test case for remove-private-as

* Added failing test case for order of bits

* Added failing testcase for submodule

* Add support for map of module name and entities for input/output of crud service (#22)

* CRUD multi-module APIs need a fix on core side to work
 * Added more test cases to sanity tests. some are not working and need fixes

* Added better suppport for enums to work with unions and leaf-list (#23)

* Introduced Enum::Value to represent enum values
 * Introduced ValueList to represent leaf-lists
 * Added unit tests for these

* Add support for leaf-list of bits + test case + clean up bits

* Fix boolean read

* Codec entity (#24)

* Addded entity lookup files

Added entity codec

* Updated test case

* modify cmakelists.txt to download libnetconf and libyang

* Clean up core and test ydk cpp against XR (#25)

* Disable support for 4 XR native models
 * add unit test for netconf provider
 * Add separate class for netconf model provider

* Revert "modify cmakelists.txt to download libnetconf and libyang" as it was breaking tests

This reverts commit a23627f.

* Renamed core netconf test and added tests for XR

* Update CMakeLists.txt

* Add 0.5.1 bundle profile

* Apply Xiaoqin's fix for inter-bundle header include

* Make bundle package non-debug

* Same name fix (#26)

* Header fix

Fix header name in entity_lookup.cpp, API belongs to other bundle
use full name, "ydk_<bundle-name>/<API-name>.hpp", as its header name.

* Added underscore suffix to nested containers with same name

* cmake header install issue resolved

* Added global variables, code review improvements (#27)

* Temporarily comment out failing tests

* removing dependency on builtins str

* remove dependency on six

* Remove cisco-ios-xr bundle dependency on openconfig (#237)

* Add bundle install details to README

* Add edits to README for readability

* Fix README typo

* updating install.sh to create python 3 env and readme (#242)

* update readme

* update install.sh to create python3 env

* update readme with note about python envs

* Fix minor typos in README file

* Fix oc-if sample app (#246)

* Fix Enum references in oc-if sample

Sample for oc-if model was using old-style enum names (*_Enum). These
changes make the names consistent with the current name style.

* Make oc-if sample app executable

* Update travis (#260)

* Add final bundle adjustments for next major release (#258)

* Update model bundles for YDK 0.5.1

Updates models in cisco-ios-xr, ietf and openconfig bundles. The
cisco-ios-xr bundle now includes XR 6.1.1 models. Bundle version is now
aligned with XR release number.  The openconfig bundle now uses github
files exclusively.  The ietf bundle now includes model for syslog types.
The versions of the ietf and openconfig bundle is bumped to 0.1.1.

* Fix IETF bundle version

Bumping bundle version to 0.1.1. Bundle file was incorrectly using
bundle version 0.1.0.

* Add OC deviations for cisco-ios-xr bundle

* Rename initial bundle files

Initial bundle files now include version details in their name.  This
nomenclature facilitates the accumulation of bundle files over time, so
a particular bundle version can be re-generated at any point.  This
nomenclature is also consistent with how profile files were named before
the introduction of bundles.

* Remove XR deviations for OC interfaces

Deviations for OC interfaces break cisco-ios-xr bundle generation.
Deviation file is being removed from the bundle definition and a issue
filed (#259) for resolving the dependency appropriately in the next
release.

* Issue 216 (#261)

* Added weak "internal use" indicator

* Only print import identity statement used (#216)

* fix if statement

* Update version to 0.5.1 (#262)

* Update ydk.services.rst (#263)

* Removed auto cap (#265)

* Added datastore option check (#252) (#268)

* Added datastore option check (#252)

Disabled delete config test case in test_sanity_netconf.py because ConfD
does not support startup and candidate datastore at the same time.

* Code review change

* Removed unnecessary import statement

* CRUD delete on list with id ref as key bug fix (#270)

* Update samples/readme for 0.5.1 (#271)

* Loggin fix (#269)

Removed duplicated logging(#251), added loggin reply for get/get-config(#254)

* NetconfService Issues (#248), (#247), (#235) (#274)

- Updated documentation and docstring to NetconfService (#239)
- copy-config/validate now can accept YDK entity as source (#248)
- merged commit/commit_confirmed into one method (#247)
- make copy-config/validate consistent with RFC 6241 (#235)

* Update README.rst (#275)

* Fix cmake file for bundle

 * Move types to types.cpp
 * Move errors to errors.cpp
 * Fix identity
 * Temporarily remove bgp-oper, mpls-te-oper, ospf-oper, infra-policymgr-oper

* Fix large model files issue by using compiler optimization -Os

 * Add back the temporarily removed large models to cisco-ios-xr bundle profile
 * Generate/compile time down to ~36 mins on mac

* Core doc (#17)

* Migrated some static docs from core.hpp

* Update static doc

* tweak

* Added "Edit on GitHub" link, used forked branch temporarily.

* Added doc for crud service and netconf provider

* Code review update, disable "edit on GitHub" link for generated docs

* update iskeyword bug

* fixed generate module path bug

* Test case fix, change afi_safi_name to Identity

* Fix deviation table key

* Apply Abhi's C++ patch

Apply Abhi's C++ patch for compilation and test cases in ubuntu
environment.

* Added C++ tests to CI

- Updated .travis.yml, test scripts and some sanity test cases to
  enable execute same tests in osx and linux.

- Reorganized sdk/cpp/ydk/tests/confd, modified original Makefile
  under this directory, and added three directories: augmentation,
  deviation, and ydktest, that contain Makefile for different ConfD
  environments.

- Moved deviation YANG models from profiles/test/deviation to
  yang/deviation, a profile file profiles/test/deviation.json could
  be used to generate API for it.
@ghost ghost closed this as completed Jan 30, 2017
@ghost ghost reopened this Jul 25, 2017
@ghost
Copy link
Author

ghost commented Apr 23, 2018

below approach can be used

git clone https://github.com/CiscoDevNet/ydk-gen.git
cd ydk-gen
pip install -r requirements
 
$ ./generate.py -h
usage: generate.py [-h] [--bundle BUNDLE]
                   [--adhoc-bundle-name ADHOC_BUNDLE_NAME]
                   [--adhoc-bundle ADHOC_BUNDLE [ADHOC_BUNDLE ...]] [--core]
                   [--output-directory OUTPUT_DIRECTORY] [-p] [-c] [-g] [-l]
                   [-v] [--generate-doc] [--generate-tests] [-o]
 
Generate YDK artefacts:
 
optional arguments:
...
-o, --one-class-per-module
                        Generate separate modules for each python class
                        corresponding to yang containers or lists.
 
./generate.py --bundle profiles/bundles/openconfig_0_1_5.json --python --one-class-per-module

Import:

from ydk.models.openconfig.openconfig_bgp import Bgp

changes to:

from ydk.models.openconfig.openconfig_bgp.bgp.bgp import Bgp

@ghost ghost closed this as completed Apr 23, 2018
111pontes pushed a commit that referenced this issue Jul 21, 2020
Merging 0.8.4.post1 to private master
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant